The word 'toilet' is often used in some form on public signs here, and common alternatives are 'ladies'/'gents', or male and female pictograms, with the latter two being more common in more up-market environments. Toilet is also commonly used in the names of and advertising for toilet cleaner, toilet brushes, toilet paper etc, and is considered perfectly acceptable for use in day-time TV adverts. There's also eau du toilette (toilet water), which is a weak perfume that I think is supposed to be applied over the course of an evening as the original, stronger application of perfume wears off.
Lavatory might be a preferred alternative to toilet among certain elite social groups, and the usage of lavatory vs toilet between social groups might have been what Vivi was referring to, but toilet has a very high degree of public acceptance, and using it wouldn't mark someone as filth unless you take 'filth' as encompassing >99% of the population.