My Butterfly Garden

My butterfly garden is chock full of wildness. I'm afraid I'm at a loss as to what to prune out. What I think are weeds turn into magnificent primroses and other specimens. If anyone can tell by looking, let me know and I'll pull out the offending stalks. All winter it looks awful, but finally I am rewarded with color. I do know the tall brown stalk needs to go, but how about all that other unidentified stuff? Is it supposed to be that way? Is it safe to tug it out? I get some yellow flowers soon, I know.

Here are some samples.

Coral honeysuckle - identified by some Facebook/art friends.

Clematis and primrose.