A cutting stylus is a very small "needle" that carves out the groove in a record.  There are many different types.

Microgroove vs Standard (78rpm)
Standard (78): The first lathes utilized a cutting needle with a 3mm tip webmail.  These grooves were much wider than on modern records, and therefore had to be spaced out farther on the disc. 
Microgroove: Sometime in the late 40's, a new standard emerged called "microgroove" which used cutting needles with a 1mm tip.  This allowed for much longer sides to be cut. Some microgroove records will have two lines etched into the base of their shank.

Materials:
Plotter blades - (Experimental and for embossing), Plotter blades are originally used with Cutting plotters.

Steel - Steel needles were most often used on home recording units with magnetic heads.  They were of relatively low-fidelity and had a very short cutting life-span.

Stellite - As described on Wikipedia  : "Stellite alloy is a range of cobalt-chromium alloys designed for wear resistance". Stellite styli are slightly better than Steel ones

Sapphire/Ruby needles - These are the industry standard cutting needles, and are still made by Apollo Masters. Rubies are red, and Sapphires are clear, but they are essentially identical. These allow for much longer cutting life-spans and much higher fidelity.  They are, however, extremely fragile, and easy to break. Average cost for a new ruby needle is around $90.

Diamond - Diamond styli are extremely expensive and rare. Many uninformed people mistake clear sapphire needles for diamonds, and will list them as such on sites like ebay. However Diamond styli were only made for a couple of specific applications.  They were initially used exclusively for Direct-To-Metal Mastering and were used to etch into special copper blanks (Stylus developed by Teledec in Germany then Mr Lehman in Berlin and now Indimant bough the knowledge)).  More recently they have been proposed by Vinylium (swiss) and Vinyl Recorder for their "Dub Cutting" machines to cut plastic blanks.  

One company is specialized and is still developing diamond cutting stylus (DMM - PVC - Prototypes)  in France : My Shank S.A.S.

Shank Type - There are several different types of needle shanks available.  The shank is the metal piece that is attached to the sharpened stone and is held in place by the cutter head.  Each cutter requires a specific type of shank.