When I started out in the SCA armor was a helm, hockey gloves, elbow and knee pads, and a cup. If you're dueling with your friends, with poly blades, you're probably not going to need more than that. A nice padded jacket maybe. When something gets hit enough to hurt, put new armor on it!
Just remember, a little common sense and playing nicely is your best protection.