What do you like best?
It's free and opensource, has build-in Wiki, Documentation, File storage.
It was written on RoR, so you can easily extend it with any feature you want, and there are many many ready to use plugins. And RoR support a lot of databases out of the box, so you are not stricted only to MySQL or PgSQL. RoR and hence Redmine support REST API, which made all interaction with Redmine much easier.
Moreover redmine is a good tool not only for issue tracking but for project management as well with build-in Gantt diagrams and Versions control.
Redmine is really well known, so you probably won't have any troubles to attach any of your service to the Redmine installation, import or export tasks etc.
Redmine has good support of many VCS like not only Git and SVN, but Bazaar, Mercurial and more...
What do you dislike?
Frankly speaking the interface is pretty outdated, they have to update it long long time ago. Redmine has themes feature, but it's not about how you interact with the Redmine, but more like how it looks like (different colours, buttons shape etc.)
One more feature I'd like to see there is ticket prefixes like in Jira, right now all tickets has unique numbers which are assigned through all Redmine, not per project.
One more drawback that Redmine is pretty slow in serving huge amount of simultaneous connections. It's ok if we have 50-80 developers and admins working with it all day long, but if we have more than lets say 200 of users, redmine should be tuned up in terms of performance.
Recommendations to others considering the product
You should definitely take a closer look on Redmine. It's just a brilliant in many many facets. Of course there are things to be improved, but in general I enjoy experience with Redmine.
What business problems are you solving with the product? What benefits have you realized?
Tracking issues, keep all information about problems together, creating documentation, tracking changes... Right now I'm using it as my personal GTD tool, and it's pretty useful.