Right you are, though most nations take their deportees (who are often deported for criminal records). There are very few "recalcitrant" countries that refuse.

And, the immigration law allows deportations to alternate countries if the country of origin won't accept them, though that rarely happens - I've heard of that happening once in the past 25 or so years of practice