Divide your guests into groups: immediate family, closest relatives, extended relatives, family friends, friends, acquaintances, kids, coworkers, etc. Once you've both classified your lists, see if you can trim the list by removing entire categories. Maybe you can both nix the young kids, the acquaintances, and the coworkers. Keep going until exceptions start to pop up, and then evaluate each possible guest individually.
Photo by Mi Amore Foto via Style Me Pretty