Coming from a 25-member tech team where all local members are very close (we get along well), I think that a big part of what makes us operate well as a team is our closeness. We get less stressed during crises, communicate better, etc. because we are connected.
That type of connection is difficult to establish with our remote members. Because they aren't sitting right next to us, we can't joke around or come over to chat with them in order to get some mental relief from the daily grind.
Essentially, our work environment is enhanced by personal contact with fellow employees, something that isn't easily established with remote workers.