Things to consider about fruit and your diet:

1. What does your carb consumption look like in a day? (Including fruit).
2. Is your body having to burn through the glycogen produced by your conscious carbohydrates AND the fructose deposited in your liver from excess fruit consumption?
3. Have you been "doing all the right things" with your fasting and NOT seeing results? 
4. Be willing to experiment with finding alternatives that provide the same nutrients as your fruit and see if lessening or removing fruit consumption changes how your body is responding to IF.