A host with address 10.10.10.10 and mask 255.255.255.0 wants to send to 10.10.20.5. What does it do with the mask, and what does it conclude?
Answer
It ANDs both addresses with the mask, gets 10.10.10.0 and 10.10.20.0, and sends the packet to its default gateway
The AND test is applied twice, once to each address, and the results are compared. Here they differ — 10.10.10.0 against 10.10.20.0 — so the destination is not local and the packet goes to the default gateway. The second option gets the mechanism right and the arithmetic wrong; with a /24 the third octet is part of the network, so 10 and 20 are different networks. Comparing octets by eye and stopping at two happens to be how people guess and is exactly what the mask exists to replace. ORing with the mask is a different operation used to derive a broadcast address, not to decide locality.