Wrong. He should know.

She has no right to privacy in such things while he's away.

What if it came out that you knew and didn't tell him?

It's a difficult situation, but it was created by the wife. The way to face difficult things is to face them, not deny them. Maybe she'll make an excuse to him and get a new respect for the eyes of the townspeople (she doesn't need to know how he knows) and go on the straight and narrow. Or he'll decide to stop working himself to death to support this woman. Either outcome would be preferable to his not knowing at all.