Drain peaches, reserving 1/2 cup syrup. Tie spices in a small piece of cheesecloth. In saucepan, combine reserved peach syrup, vinegar, sugar, orange slices, peaches and spices. Heat to boiling and simmer uncovered for 5 minutes.
Cover and let cool to room temperature. Remove spice bag. Serve peaches warm or chilled and stud with a few whole cloves if desired.