Give us a link to those exact ESCs you have.
Connect to MissionPlanner and go to Initial Parameters.
Set you prop size and battery cell count, also select “suggested” settings then accept everything it offers.
Do not skip any settings, it is a mistake to think you dont need all the battery failsafe settings, or the Fence, while testing these larger copters.
Then set these:
ARMING_CHECK,1
INS_ACCEL_FILTER,10
INS_HNTCH_ENABLE,1 ← set this then refresh params to see the rest
INS_HNTCH_MODE,1
INS_HNTCH_REF,0.2
INS_HNTCH_FREQ,40
INS_HNTCH_BW,20
INS_HNTCH_FM_RAT,0.7
INS_LOG_BAT_MASK,1
INS_LOG_BAT_OPT,4
LOG_BITMASK,180222 (added PID)
Depending on that ESC model, there might be more to do before another test flight, so stop at this stage for now.
After we’ve verified the ESC details, do a hover for a while in AltHold, just with some gentle movements and lets see how that goes.
EDIT:
I would probably set these anyway, regardless of the ESC model since HobbyWing seem to be shy about supplying specifications lately.
MOT_PWM_MIN,1100
MOT_PWM_MAX,1940
Do the semi-automatic ESC calibration:
https://ardupilot.org/copter/docs/esc-calibration.html#semi-automatic-esc-by-esc-calibration
Then you will have to use MissionPlanner motor test to check and set MOT_SPIN_ARM for the lowest reliable, smooth start of motors.
Then set MOT_SPIN_MIN by adding 0.03 to the value you found for MOT_SPIN_ARM.
Now you can do the test flight, and allow some time for MOT_THST_HOVER to relearn. You may have to adjust the throttle quite a bit to keep the copter hovering, and it should settle down and hover by itself once MOT_THST_HOVER has sorted itself.
This isnt so much tuning as just gathering more data, so dont expect any great improvement in attitude control yet.