Field groups are used to visually gather several fields that are associated by some sort of logic, for instance several text fields that hold the different parts of an 'Adress'. On input forms and displayed content, the corresponding fields are enclosed inside an HTML fieldset.

At the bottom of the Manage fields page for a content type, you'll find this:

In order to add a new group to a content type, you need to provide the following information:

Label

A human-readable name for the group. It will be used in input forms and on displayed content.
All characters are allowed, including spaces, accentuated or non-european characters.

Group name:

A machine-readable name for the group. It is used internally to identify the group. When doing custom theming, it is this identifier that you'll use to refer to that group.
Important: The group name cannot be changed once the group has been created.
Allowed characters: a-z (unaccentuated), 0-9 and the underscore (_).
The length of the group name cannot exceed 32 characters (including the 'group_' prefix that gets added automatically - that is, 26 free characters)

Once a group has been created, you can define what fields it will enclose by rearranging fields and groups.