The lowest you can go is your essential body fat, which is vital fat mass. Never go below 8-12% for women and 3-5% for men. Ideally you want a little more than this unless you are an athlete or body builder. Fat has many critical roles in the body including hormonal balance and building the brain and nervous system. A lack of fat in the diet can lead to mood imbalances and for women when body fat gets too low it can lead to infertility and loss of periods.