In many cases, recognition of your degree is not necessary. But there are some professions for which you need to go through a recognition process, for instance, doctors, lawyers, nurses or engineers. These professions are called "regulated professions" ("reglementierte Berufe"), and you must have your degrees recognised if you want to work in them. Furthermore, you need to have your professional qualifications recognised if you want to become self-employed in a so-called "licensed trade" ("zulassungspflichtigten Handwerk"). You will find a list of all the licensed trades which require recognition in the Trade Guild Codes.
If your job is not regulated, you do not need to have your degree recognised. Going through recognition, however, improves your chances of finding a footing in the German job market. On the Employment Agency's website (in German) and ec.europa.eu (in English) you can check whether your degree is part of the regulated professions in Germany.
Please note: If you would like to come to Germany with a job-hunting or employment visa, you need to have your qualification recognised in advance.