Either when all related or not.
we've never vacationed with another family but often call visits to see family vacations, Lol.
Can't quite imagine the logistics. Besides, I have never been able to get dh to take a vacation with us, so not sure how I would get it to work. My brother wanted to do it one year, but I had to tell him that although I would try to join his family for a few days, I could not see a way to making the other two participate.
When I was a kid my parents did this all the time with their friends (with children).
“Clearly," said Arthur,"you're an idiot- but you're our kind of idiot. Come on.” ― Markus Zusak, The Book Thief
If everyone gets along and has a similar circadian rhythm, it would probably work.