The Case

1. The most obvious one the computer case. Some, such as the iMac, are actually integrated into the monitor screen. As such, replacing components would be much more difficult. The advantage of this "all-in-one" computer is that it's easy to transport and it cuts down on the number of cables lying around. On the flip[ side these are terribly expensive to repair as it requires special skills and tools to take them apart.

   The Power Supply

2.  
   The Power Supply (PSU) will typically have a sticker on it showing the power rating.
   Most of us, however, have a case that the monitor plugs into. Most cases that you buy off-the-shelf include a power supply unit. This PSU has a power rating measured in watts. For modern computers(dual and quad core machines) a PSU with at least 500 watts is strongly recommended -- 650W just to be on the safe side. This gives your computer room to grow. The common theme is that a more powerful component, while more expensive, will not need to be replaced as often. However, the price of the most powerful component of that type is usually much more expensive than the second-most powerful component.

   Storage Devices

3. Your PSU supplies electricity to all the components in the case. Inside this case, you are probably going to have an optical drive to read and write CDs and DVDs. You will also have a storage device referred to as a "hard drive." Both an optical drive and hard drive are connected to the motherboard with data cables(older systems had IDE, new ones have SATA cables), in addition to being attached to the PSU.

   Key Components

4. You may also have a video card that sends all the visual data to your monitor screen. Sometimes this video card is integrated into the motherboard. If it is not, you may have to connect a PSU cable to it. Sometimes you will have a sound card and a network card, but these are more commonly integrated as well. If all of these components are integrated, their external connectors will be in the back of the motherboard (on the rear of the case).
   NOTE: High performance computers usually used by "Gamers" have separate video, audio and network cards.

   CPU and RAM

5. 
   Here is a CPU without the heatsink/fan combo attached.
   The central processing unit (CPU) plugs into the motherboard. This processor will have a heatsink and fan on it to keep it cool. Sometimes you can purchase a CPU without a heatsink/fan combo, which is called an OEM (original equipment manufacturer) purchase. This option is not typically found at a retail store, which will instead supply a heatsink/fan combo already attached to the CPU. Most computers come with an aluminium heatsink while high end systems use Copper which is better at dissapating the heat from the cpu.
   
   RAM (Random Access Memory), which your computer uses to rapidly store and retrieve temporary data, comes in "sticks" and plugs into long slots on the motherboard. Consult your motherboard manual to find out what CPUs and RAM is compatible with it. Mixing ram is not good for your system. There are 2 types of ram, high density and low density. The best way to explain this is to "Google it". If you need new ram then I prefer going to http://www.crucial.com/index.aspx where you can find exactly the memory you need.

This should give you a basic overview of what your computer is about. If you have questions then please go to Ehow.com or Answers.com.