Territory refers to the ownership of a piece of land. Territory can be neutral or under the control of a nation. A nation's territory is any piece of land inside that nation's national borders.
Allied territory counts as own territory in most cases - you may construct buildings and even cities inside allied territory and you will gather extra resources from connected rare resources after researching Taxation at a Temple. Furthermore, military units can move through allied territory without suffering from Attrition.
Building
A nation may build new cities, military structures as well as resource-gathering structures only in it's own territory (= within it's borders). An exception to this are the Lakota introduced in the Thrones & Patriots expansion pack - they may also build in neutral territory.
Building Cities or Forts extends a nation's borders, thus increasing the territory controlled.
Generating resources
Researching Taxation technologies at a Temple allows a nation to generate 
Wealth based on the percentage of world landmass controlled.
Also, the Mongols automatically gather Food based on their territory. They receive +1 
Food for each 1% of world landmass controlled, times half the number of nations. E.g., 25% of the world landmass when playing with a total of four nations would result in an extra +50 Food.
Rare resources
The income generated from rare resources can be increased by researching Taxation technologies or building the Porcelain Tower wonder. However, this only affects resources within own or allied territory - resources in neutral or enemy territory won't benefit from this.
Artillery units
Siege and artillery units outside of their own nation's borders are considered 'out of supply', if not within the range of a Supply Wagon: They reload and fire at a much slower rate, with early ages siege units (Catapults and Trebuchets) reloading at 66% of normal speed and artillery units from the Gunpowder Age on reloading at 50% of normal speed.
When an artillery unit is selected, a small icon in the status bar indicates, if it's 'in supply' or 'out of supply'.
Attrition
If a nation has researched Attrition at a Tower, entering that nation's territory will cause other nation's units to suffer attrition damage, if not in range of a nearby Supply Wagon or a Despot, Monarch or Comrade patriot unit.
In addition, units entering a neutral nation's territory (The units' parent nation is at peace with the nation whose territory is being crossed) will suffer severe attrition damage, regardless of supply units within range, or technologies researched by either nation. The only way to remove the attrition penalty is by efectivelly terminating the diplomatic neutrality: Either by forming an alliance, as allied nations can cross each other's territory without suffering attrition; or by declaring war, in which case all the common rules of attrition apply.
Note that attrition only affects land units, with the exception of Scouts, Spies, and Supply Wagons; and with Thrones and Patriots, Patriots with Supply bonuses (Despot, Monarch, Comrade). Therefore Ships, Missiles and Aircraft are never affected.