Yes, but set realistic expectations. Listings on the big portals can be compared in seconds, and many show a price rating against similar cars. Dealers know this and often price close to the market from the start. Large discounts are the exception, not the rule.
What you can usually achieve is a modest reduction or extras that have real value, such as a fresh HU, a service, new tyres or a longer warranty. A reduction is most likely when you have a concrete reason: a defect, a service that is due, an HU that expires soon, or a car that has been listed for a long time. For the market overall, see buying a used car in Germany.