Grammar doesn't have to play an outsize role in your life for you to wonder if you're using the right version of a word. So: should we have said "outsized" instead of "outsize"?
Well, maybe. Or not.
The adjectives outsize and outsized have the same meanings ("unusually large or heavy" and "exaggerated or extravagant in size or degree") and they appear in the same contexts; they're technically what we call variants. (Outsize alone also functions as a noun—it means "an unusual size" and especially refers to "a size larger than the standard," as in "a store specializing in outsizes"—but the noun is newer than both of the adjectives, and not to blame for either one.)
It's easy to find examples of both variants:
All this attention on a platform that's not that widely used may feel outsized, but that's because its influence on publishing is gigantic: Twitter is the platform that led us into the mobile Internet age.
— Adrienne LaFrance and Robinson Meyer, The Atlantic, 30 Apr. 2014There's an outsized oak flavor and resin to deal with on the finish, but overall this captures the essence of Ribera del Duero for under $20.
— Wine Enthusiast Magazine, June 2013For such a modest creature, the two-inch-long snapping shrimp creates quite a racket. Also known as the pistol shrimp, the crustacean possesses an outsize claw that it cocks open and snaps shut at lightning speed.
— Catherine Zuckerman, National Geographic, July 2011Instead of having actors perform with outsize props, Ant-Man is utilizing both macro photography—the filming technique used on bug documentaries—and motion-capture technology.
— Clark Collis, Entertainment Weekly, 16 Jan. 2015
Both date to the early 1800s, though outsize appears to be a decade or so older. It was at one time the preferred variant throughout the English speaking world, but in the late 20th century American English took a shining to outsized and that variant has been the favored choice on this side of the pond ever since. In British English outsize continues to be preferred.
We will likely never know why one variant gained favor in the U.S. while the other held fast in the U.K. Perhaps American English speakers liked outsized because it seems more adjectival in the way that words like charmed and embarrassed are. (Those of course both come from verbs, while the size in outsized is the noun.) If that's the case, it's interesting that the same appeal didn't translate for British English speakers.
You can use either, no matter where you live. Your audience will more likely than not understand you perfectly well, and this is not one of those issues that gets a lot of loud complaints. So go ahead and pick one, and don't make an outsize deal of it.