<uses-permission android:name="android.permission.BLUETOOTH" />
This makes Google Play decide that my app only works on devices that supports bluetooth and I've believed the only way around this would be to create another app with the above line removed from the manifest.
Yesterday I realized this is not needed when I learned about the <uses-feature> tag. It turns out that Google Play translates the permissions your app request in to features required like this table describes. So my <uses-permission> tag above makes my app require the android.hardware.bluetooth feature.
With my app bluetooth hardware is not needed, but my app needs that permission on some Android versions in order to detect when a bluetooth keyboard is connected or disconnected. So what I needed to do is to add a line to my manifest to make this feature requirement optional.
<uses-feature android:name="android.hardware.bluetooth" android:required="false" />
This tag can also be used to add a requirement by setting the required flag to true.
When publishing the new version to market, the number of available devices went from 1265 to 1288 and I've already got it confirmed from a user of the app that it can now be installed on a non-bluetooth-capable device.
In my case this was all that was needed but depending on how your app uses the optional features you specify you probably have to use the hasSystemFeature call to check if the feature is present or not.