X = 4 and Y = 13. S knows the sum is 17, and P knows the product is 52.

S makes a list of all the addends that could produce 17 and fulfill the other conditions above:

2 + 15 (product: 30)
3 + 14 (product: 42)
4 + 13 (product: 52)
5 + 12 (product: 60)
6 + 11 (product: 66)
7 + 10 (product: 70)
8 + 9 (product: 72)

S knows that P is thinking of one of the products listed above; she can say that P doesn’t know X and Y because each of these products has multiple solutions (e.g. 30 = 2 × 15 = 3 × 10).

P has been building his own list — in his case, he’s listing pairs of factors that produce 52. It’s a short list — there are only two possibilities that meet the conditions:

2 × 26
4 × 13

When S makes her first announcement, P understands that her candidate list is like the one above — it’s one in which all of the products are ambiguous. And this eliminates the first candidate on P’s list. If X = 2 and Y = 26, then their product would be 52, as P expects, but S would know that their sum was 28, and she couldn’t then claim that P didn’t know the numbers — her list of suspects would look like this:

2 + 26 (product: 52)
3 + 25 (product: 75)
4 + 24 (product: 96)
5 + 23 (product: 115)
…

That fourth item, X = 5 and Y = 23, gives a product of 115, which has no other set of factors that meet the requirements. So if 28 were the sum, S couldn’t have declared so confidently that P was still in the dark.

Realizing this, P can eliminate 2 × 26 from his list, leaving only one possibility: X is 4 and Y is 13. He says, “Now I know X and Y.” And by reconstructing his thinking, S can determine these values herself.

(Thanks, Jon.)