I tend to design games to encourage a particular playstyle, rather than to repel one. Mostly.
Most of this comes down to the mission statement that defines the whole game design (and gameplay) process. I will be very clear about what the game is and isn’t there, in hopes that people will self-select whether they’ll like the game or not. Policies are the next step, being very clear what will be encouraged and disallowed (PvP rules; alt rules; IC and OOC harassment rules; social contract; plotrunning rules; how real-life prejudices will be handled in a modern, historical, or near-historical setting; etc).
My general workflow is to determine what kind of RP I want to see on the game, how to support and encourage that, spot and remove any roadblocks to it (the waygate system in The Fifth World is my best example of that, we had a multi-planetary game, but didn’t want barriers to casual RP, so we added limited instant travel in so folks could be on whichever planet they wanted), and then let players know what the intended thrust of RP will be.
That being said, most of the choices of what I want to play: PvE, zero-tolerance-for-harassment, collaborative playstyle, combination of action and casual, clear reasons for people to work together (with IC friction), lend themselves toward the type of players that I enjoy playing with.