Today's nuclear plants don't use lithium as a coolant. They use water. They do use lithium as an oxygen scavenger in the water though. However, in pressurized water reactors, there is a small amount of lithium produced in the reaction of boron in the water.
I've read that future plants may use lithium, especially breeder reactors, but those are still experimental.