There are several different levels to this optimisation and I will go into each one in turn.
Use the Correct Image Format
There are many different image formats that can be used on the web. Each has their advantages, disadvantages and their correct use on the web.
GIF - Graphics Interchange Format
Introduced in 1987 by CompuServe, the GIF image format has become synonymous for gaudy animation as it is the only web image format to support multiple frames. It is an 8-bit format which means that it can support up to 256 colours and transparency.
GIF images should be used when you need low-resolution images with or without transparency. GIF images are not compressed, so the quality remains constant. They are mainly used for logos and graphics with solid areas of colour.
Who remembers the dancing baby from the mid-90's?
PNG8 - Portable Network Graphics
PNG8 was created as an improved, non-patented replacement for Graphics Interchange Format (GIF), and is the most used lossless image compression format on the Internet. As with the GIF format, PNG8 is an 8-bit format and supports 256 colours with transparency and it is uncompressed.
GIF and PNG use is almost interchangeable, however PNG files generally has a smaller file size when saved.
PNG24 - 24-bit Portable Network Graphics
PNG24 is an improved version of PNG8 which supports 24-bits. This means that it is not limited to 256 colours and uses a true colour palette. PNG24 is the only file format to use a true alpha mask, as opposed to single bit transparency. Single bit transparency is where a pixel is either a colour or transparent. Alpha mask means that a pixel can be any colour, and also a percentage of opacity. This gives a much higher quality image, but at the expense of higher image formats.
PNG24 should be used when the alpha is required, such as logos, icons and any other image that requires alpha transparency.
JPEG - Joint Photographic Experts Group
JPEG (jpg or jpe) is the method of choice for serving photo quality images on the web. It uses a compression algorithm which removes data from the image in order to reduce the file size. It also compresses similar parts of the image together. The higher the compression level, the more aggressive the algorithm removes and compresses data. You should find a trade-off between image size and image quality. If open a JPEG image in an editor, make a change and re-save it, you will loose more data from the image. The more times you do this, the more data is removed by the compression algorithm. Eventually, the image will be so garbled, you won't be able to make anything out.
SVG - Scalable Vector Graphics
SVG is a relative newcomer to the graphics format. It is the only vector format in this collection. SVG is an XML-style markup driven format supported in every browser, except Internet Explorer versions prior to 9 and Android versions below version 3.
SVG offers resolution independent graphics, which means that the image can be enlarged or reduced without pixelation. SVG offers a way to do full resolution graphical elements, no matter what size screen, what zoom level, or what resolution your user's device has.
SVG is a revolutionary image format, but it is not suitable for complex images or photographic images. It is great for logos, symbols, icons, graphs or anywhere else where solid colours or basic image elements are required.
Regardless of the file format chosen, all images should be optimised. If you use Photoshop or GIMP this can easily be done by using the "Save For Web..." export option. This will not only allow you to select the most appropriate image format, colour depth, transparency and so on, but it will automatically strip out surplus hidden data that is not required.
This surplus data is known as metadata and is mainly stored in JPEG images. It includes information about the camera that was used, the focal length, shutter speed, exposure settings and also the colour profile used. This extra information increases the image size by over 100 Kb and is generally never seen.
If you do not have Photoshop, you can use the online image service Smush.It from Yahoo which will do the same process. Simply upload your images, they will process the images and let you download an archive with the optimised images.