There are two issues to think about when we consider the relationship between Buddhist teachings and guilt:
1) Buddhism is primarily a preventive approach to wrongdoing, rather than a curative one. This is wise: much better to prevent bad things from happening than to have to deal with them once they have happened (a failure to recognize this is, maybe, the main problem with modern medicine, for example). Someone who cultivates the discipline to follow the Eight-Fold Path will, to the extent that they succeed, succeed also in doing no wrong. Basic Buddhist teachings are primarily aimed at cultivation of right being. They do not tend to dwell on atonement and expiation.
2) The First Noble Truth of the Dharma, that all life is suffering, refers of course to all forms of suffering. But as a human cultural artifact, Buddhist teachings over the centuries pay particular attention to mental suffering: negative thoughts and feelings. I do not wish to digress into a discussion of variants on Buddhist teachings here (trying to be brief), but it is worthwhile to point out that Tibetan (Tantric, Mahayana) Buddhism stresses the essential goodness of human nature and aims at liberation through positive self-realization, while Zen (Chinese, Taoist) Buddhism stresses the non-existence of the self and aims at liberation through selfless mindfulness (I have no intention here of judging between the two or even claiming that they are fundamentally different: I neither favor any school nor claim that there are ultimately fundamental differences). All Buddhist teaching aims at the end of ego-suffering through identification with the whole world (nirvana).
What I want to think about is the problem of real guilt. That is, for the sake of discussion, let's assume that one has in fact acted wrongly: willfully transgressed one's own moral principles. It is not a question of being "made to feel guilty" in some illegitimate way, and it is not an illusion of the ego, as when the ego leads us to think that negative outcomes are a result of our actions as a way of making us feel significant. (And perhaps there are other examples of false guilt.) No, let's assume that we are actually guilty. It's not impossible!
Buddhism teaches us that only ourselves suffer from the negative thoughts and feelings that we are experiencing. This is one of the most important insights of Buddhist psychology. Say someone negligently ran over my foot with their car and broke it. They may be rightly called upon to pay for the treatment or some civil responsibility like that. But they, the negligent driver, are hardly the right person to help to heal my injured foot. I will need a doctor for that, and above all I will need to follow the regimen, practice the physical therapy, and do everything necessary to cure my foot. That will be wholly my own responsibility.
Negative thoughts and feelings are like the injured foot. It is not a question of whether the negative thoughts and feelings are understandable or even justified. That is besides the point. The point is that it is I who now carry around the negative thoughts and feelings, that repeat themselves in a "crazy mind" tape loop in my head, just as the suffering of the injured foot persists until it is dealt with. It is I who am suffering, and so I must somehow overcome and lose the negative thoughts and feelings.
But notice that that discussion is from the point of view of the innocent. Today I want to think from the point of view of the guilty. And so we can see a potential danger if we misunderstand Buddhist teaching: granting that it is possible to be guilty, which I take to be a plain fact, we do not want to become so proficient at clearing the mind of negative thoughts and feelings that we lose our conscience altogether. That would be a grave misunderstanding of both Buddhism and Taoism. But at the same time it achieves nothing for the guilty person to be masochistic: to say to their self, "Yes I deserve these negative thoughts and feelings - I deserve to suffer." That by itself only makes the world worse, not better.
For ourselves, we can learn from our transgressions. We can meditate and become more mindful of the necessity of right action, right speech, and the other elements of the Path. There is nothing magical or mysterious about this. There is nothing mysterious or magical about Buddhism, at all; that is a very important point. For others we have wronged, I have only humble suggestions:
1) Atonement through concrete actions of restitution, when possible. Quotidian examples: returning stolen property, repairing or paying for damage, admitting lies and telling the truth. Replacing antagonistic actions with supportive actions.
2) Apology. But the act of apology is not without some risk of seduction by the ego. Perhaps further contact with you will only prolong or exacerbate the suffering of the person you have harmed. Perhaps your apology is a selfish act: perhaps it is only for your own sake that you want forgiveness. (Slanderers: the only person who does not need to hear from you is the person you have slandered. It is everyone else who you must speak to now.) Better to show contrition through deeds, and remember that non-action is often the best path. Assuming that you are the cure is just as egotistical as assuming that you are the disease.
This is not much to offer after bringing up such a promising topic, I realize. In the end I think that Buddhist practice is preventative, as I said at the beginning: cultivation of spiritual discipline and mindful character should aid us in avoiding bad action. And what would we call someone who did no harm? We can make up words. We'll never meet anyone like that.