Can one be too dumb for programming?

I've always flunked at math; and knowing how intertwined programming is with math, I'm skeptical of my ability to learn how to code. Can someone be too dumb to learn programming? If it helps, I'm mostly interested in learning Common Lisp.