A historic mistranslation. Originally the Giants were nor large or huge people but just a name for a special tribe, the gigantes. For some unclear reason the association with large size was made somewhere in history.