HereFlow Setup Questions

I have installed my HereFlow on a small 130 mm quad, its primary purpose is to fly indoors and will almost never fly over 2 meters. Once I set everything up on my quadcopter, all hardware installed, but optical flow and range finder disabled, everything flies just fine and altitude hold is rock solid on barometer. But once i enable optical flow and range finder, Alt hold mode has a lot of problems staying in the air and the quad keeps dropping to the ground and then jumping back up into the air. The only parameters i have changed are:
Set CAN_P1_DRIVER = 1 to enable UAVCAN
Set RNGFND1_MAX_CM = 200

just to make sure i have checked that EK2_ALT_SOURCE is 0 and EK2_RNG_USE_HGT is -1.

I am sure i am missing something, but i am not sure why the above changes would effect altitude hold flight mode. If the lidar on the hereflow is good enough for good lighting indoor conditions up to 2 meters, i would love to use that instead of barometer, but the barometer is working just fine and i would be fine if the lidar was only used for optical flow. I was wondering if there were any other settings i might be missing.

Can you post a log?