What does the "child is the father of the man" mean in "My Heart Leaps Up"?

MY heart leaps up when I behold  
  A rainbow in the sky:  
So was it when my life began,  
  So is it now I am a man,  
So be it when I shall grow old         
    Or let me die!  
The child is father of the man:  
And I could wish my days to be  
Bound each to each by natural piety.

When the speaker sees the rainbow, his heart "leaps up" with joy. He was happy to see rainbows as a baby, is still happy to see them as an adult, and hopes - one day - to be happy to see them as an old man.

Thus, his childhood enjoyment of the rainbow connects his childhood to his adulthood - and hopefully, will connect his adulthood to his old age. The rainbow and the pleasure the speaker takes in it is a constant point in his life: an unchanging pleasure in a changing world.

It is because his childhood enjoyment of the rainbow connects him as a child and an adult: hence, the man he is grew from the child he was. For that reason, the child might be thought to be father of the man.