Yes, if you can carry the extra time and cost. South Georgia is a substantially different place from the Peninsula, and the wildlife density there is not matched anywhere else on the route. King penguin colonies run to the hundreds of thousands, the beaches are crowded with fur seals and elephant seals, and there are king chicks right through the visitor season because they breed on a different cycle to the Antarctic species.
The cost is real. A South Georgia itinerary is longer, usually by around a week, and priced accordingly. It also commits you to sailing, because flying to Antarctica rules South Georgia out almost entirely. There are two or three very expensive trips a year that try to combine them, and you'd still be crossing the Scotia Sea, which isn't a gentle stretch of water.
The history is the other half of it. Grytviken, Shackleton's grave, and the Fortuna Bay walk that finishes his crossing. For a lot of travellers that turns out to be the part they talk about afterwards.
If you're choosing between South Georgia and a longer Peninsula trip, the honest split is that South Georgia is for wildlife and history, and the Peninsula is for ice and landscape.