I found the culprit!
Apparently its rare for a wasp sting to get infected but mine did. The swelling had gone down somewhat and then was worse yesterday. So my husband took me to the urgent care place - my finger was red, really swollen and a little warm so I was given some antibiotics. Its a type that will make my skin more sensitive to sunlight, so I will have to be more careful going outside the next few days. Makes me feel a little bit sick too.
Oh, and I was able to confirm that it was a paper wasp. I checked - very carefully - around the butterfly bush I had been trimming when I got stung and sure enough there was a paper wasp nest there. The wasp is now dead. The nest is gone. Yes, I like wildlife but not that much!
Met a large owl