That depends on your status.
If you have applied for or already received a residence permit according to Section 24 of the Residence Act, you are allowed to work. You are also allowed to become self-employed. After you apply for a residence permit according to Section 24, you will receive a so-called “Fiktionsbescheinigung”, which confirms that you are allowed to work.
As long as you are visa-free or with a visitor visa in Germany, you are not allowed to work. If you cannot get a residence permit according to Section 24 and you meet the requirements, you can apply for a visa as a skilled worker, for example. With this visa you can also work. You can find out more about this in the "Residence" section.
If you get a residence permit according to Section 24 of the Residence Act, you are allowed to work.
As an asylum seeker or tolerated person with a "Duldung" you have to comply with a lot of rules, including in regards to work. You can read more about this in the "Work Permits for Refugees" chapter.