Question on Minority Report... Spoilerworthy.

Hey, I searched trying to see if it was brought up on an older thread as I'm sure it was, but I couldn't find it, so if this has already been aske,d could somebody kindly direct me? Thanks!

Anyways... I've seen the movie before, and I do enjoy it, but this past time I saw it and realized something didn't quite click for me. The overall plot of the film flowed together well, except I couldn't figure out how Anderton knew Lively was Agathas mother, at what point and how did he come to the realization?

I know there's a short story that the film is based off of, and as with any movie based off of literature I'm sure there's more to the story. Is it worth checking out?

Thanks for the help!