Honestly, he explicitly talks about her reaching out to touch his "scepter of passion," and he reaching out to fondle her under her night-dress. Is that not sexual? What is it then - mere curiosity? Who is to say they aren't pubescent? Who isn't to say that had her mom not gone calling for her, Humbert would have had his desire, and the episode would have passed on, as any other early-teenage fling.
That's where the book bends - the almost consummated relationship. As he says in the opening paragraphs, he has a knack for creating an imprint of a person in the back of his mind - a perfect picture. It isn't until the end of the book, that he can shake that notion, and that is what brings about the plot. It is the idea that his concept of perfection in a woman is shaped by his focus on Annabel as the ultimate object of his love.
He of course, breaks this, as he says he did in the opening chapters, by transferring the image over to Lolita, and then destroying her. That knocks out his pedophilia, and allows him in the end to propose to Lolita that they run away together, even though she has, as he described, become her mother - no longer the 13 year old girl.

