Very simple. What makes any material colored the way it is, is due to electrons and what they are doing within. Diamond is made of carbons bonded to each other only by sigma covalent bonds. All of the electrons in diamond are occupied in these Carbon-Carbon covalent bonds, which absorb only IR light, making them transparent to our vision. Graphite is made up of sheets of conjugated aromatic benzene rings. If you consider the structures of dye molecules, you will see that the more conjugated a dye molecule is, the higher the extinction coefficient. Graphite is in effect the ultimate conjugated organic system; it has unlimited pi electrons to absorb light. By the way, all these pi electrons are the reason graphite is an excellent conductor of electricity.