That’s really sad news.
I was just thinking of going out flying using QGC. My drone systems are very expensive and I can’t risk crashing because of this pathetic bug.
@srbell Is it possible for you to record a video on how you generate bug reports on your v1.1? I tried with both of my v1.1 units but was unable to generate the bug report.
It’s already quite a lot of days since you and I have reported this but there’s no update from @philip or @kellyschrock about this. The issue is really easy to recreate on Solex.
I wonder whether they even tested Solex on v1.1 before launching it since Kelly told that she only has v1.0 with her.
I literally just took delivery of the Herelink 1.1 unit a few minutes ago, so I’ll be able to dig in and find out what’s going on with this a little later today.
@Harkhka Turn on debug options. Turn on the power menu debug log generation option. Run until it locks up. Reboot. Once you’re back up and running, hit power. Tap the debug report. When it is finished, send it off. I configured the mail app to connect to my gmail which is how I can send out the report.
That’s all I know about it. I haven’t responded because I haven’t experienced one single failure since that first one where my sticks stopped responding. It’s been flawless since then.
So I found the option in “Developer Options” for getting the “bug report option” in the power menu. I tried it, but as I told you before that when I try to share it, I get the error that “Shell has Stopped”.
Quick update: I got the Herelink 1.1 yesterday and tested this.
I’m using a brand-new HL 1.1 hooked to a Cube and a Sony DSLR for the camera. I can launch Solex and view the video stream and it works fine. My understanding is that it sometimes takes a bit to see the problem, so I let the video streaming run for a while.
Sure enough, after several minutes of streaming, I saw the lockup. The shortest interval was about 10 minutes, the longest was over an hour when I finally gave up waiting on it.
So I’m glad I’m at least able to reproduce the problem. From the app level, what I see at the point of lockup is that the media decoder receiving video frames suddenly starts to get a constant stream of errors presenting on the Android side as IllegalStateExceptions with no explanatory message at the point when I dequeue a buffer to send it to process it. From the looking around I did, this appears to have something to do with bit rate, but is most likely not the problem here since bit rate is already well-established by this point. Something else is going on and I’m not sure what it is. I have not seen this behavior on a Herelink 1.0 device.
When the app locks up, I notice that the overall device is not locked up. Using ADB (a developer tool) I can shell into the device and things look largely normal at the console level. But the entire screen is unresponsive. No amount of tapping or swiping gets any response. I haven’t thought to try killing the Solex app process from the console, but will do that the next time I see the lockup (hopefully soon, it’s hard to predict when it will happen).
One workaround I have in place, which might not actually help if the screen itself locking up is the issue: When I encounter the exception I mentioned above in the media decoder, I bail out entirely, display a “Fatal decoding error” message, and immediately stop trying to stream. With this workaround in place, I did not see the error message when the lockup occurred. This could be because I just didn’t happen to see it in time, or the screen is locked up at the point where the error message would display but it doesn’t show up because the screen is locked up.
In any case, the short story is that something new is happening on the Herelink 1.1 and I don’t have a full explanation for it yet. But I am working on it.
Thanks for the update @kellyschrock
I am glad you are able to reproduce it at your end. At least we now have a confirmation that the issue is real.
Please try to use Solex without a Camera stream. This might help you recreate the issue sooner as in my case, I didn’t plug in the camera and was able to reproduce it within 2 minutes.
Also, try constantly accessing the Android interface like the touchscreen buttons or the top menu, as in most cases the freeze is occurring because of that only.
As you said that this might be an Android issue, I am not sure whether this issue can be related to the “Shell error” when trying to share a bug report (video proof on #27 reply on this thread), or related to the fast boot screen not displaying up. Maybe there’s a possibility?
I just hope you could take this on priority and find a solution to this. It’s really hard for me to convince my clients to wait after telling them that “they’d be getting one of the best GCS systems”.
I don’t get the fast boot screen either but I can dl a bug report. Did you set your adb up so you don’t have to enter its directory location in the command line? That could be your issue. This bug report is from today, but running Solex. I was trying to reproduce the freeze with QGC, which I encountered a few days back. One thing I noted even in QGC, if you lose the hot spot there’s about a 90% you’re not getting it back lol. If you actually open the hotspot up and try the radio button/slider, it says error. On top of that, you no longer have access to wifi. Wonder if it’s related?
Re: bug report problem… Try configuring the mail app on the herelink. I set mine up for my gmail account. I think the bug report tool wants something to route the report for sending it out. Once you have mail setup I think it’ll pop up for you instead of dying when you try to proceed after creating it.
Did some more testing, and I can more or less reliably get it to lock up. The weird thing is that it doesn’t seem to be a specific app lockup, but something else. When the lockup happens, I can easily tell since literally nothing on the screen works, and that’s not how an ANR happens in Android. While the screen is unresponsive, I can shell into the Herelink with adb shell and poke around via a terminal. I can see processes running, etc. At one point, it appeared that all of the app processes were gone from the output of ps, with only what appear to be kernel processes running. (That only happened once though). My idea was to kill Solex’s process once I saw the lockup and see if killing it would bring things back to normal. Turns out I don’t have permission to kill processes from the shell, so instead I ran pm clear on Solex to clear its data (which kills an app’s process automatically). I got a “Success” confirmation in response to that command, which means it worked. That is usually accompanied by the app suddenly disappearing from the screen. But it didn’t in this case. So next I ran pm uninstall on Solex to make sure its process was so dead it wasn’t even on the Herelink. That also worked, but there Solex was, still frozen on the screen. After a reboot, Solex was in fact gone from the Herelink so I reinstalled it and resumed testing.
There’s no doubt there’s a lockup happening, but I don’t think it’s an app doing it. I’m going to keep looking though, and push an update in the next day or two. The update will at least contain my workaround for cases where the media encoder suddenly starts exploding, which might be related to this issue.
It sounds more like an Android issue now since people are experiencing the issue with QGC also.
My best guess is that the system is waiting for some signal from some hardware (probably WiFi) and then hangs while it waits for it and hence doesn’t send screen updates which cause the screen to freeze. Or it could be a screen issue also that it stops updating itself?
Since the processor is the same for Herelink v1.0 and v1.1, what else changed except the screen between the 2 versions?
Maybe this calls for some deep testing on Herelink v1.1 as with this issue it’s completely unreliable.
No OS in 2023 is crude or poorly-conceived enough to be locked up by something like waiting for something from WiFi, or to have a direct connection between things like WiFi and screen updates. But the high-level idea has merit. Something isn’t getting updated. The way it’s acting (screen updates suddenly stopping and the media decoder suddenly experiencing a series of crashes) suggests some low-level interaction with the screen itself failing. One feature of the Herelink 1.1 is a new screen apparently (it’s much brighter and looks great, yay!), so some unforeseen issue there seems plausible.
But I am working on it.