Using Pi Zero, with only 4 web page assets, and the display often hangs after perhaps 20 minutes. Often the Pi green power light is flashing like mad, suggesting that the Pi has not ‘hung’, but the display doesn’t move on to the next asset. How should I go about diagnosing, and what’s it likely to be.
I thought best to upgrade, so have downloaded 2019-04-27-Screenly-OSE but no change.
Might the SD card (8GB SanDisk Ultra HC 1) be knackered or just not up to it?
Interesting. The System Info latest logs, show that it THINKS it is still working, generating asset list, showing asset, current URL, sleeping . . . .BUT the display is stuck on just one asset!!!
Any thoughts would be very welcome. Pi power light still flashing furiously.
Hi @RobertE! Can you try to upgrade to the latest development version?
bash <(curl -sL https://www.screenly.io/install-ose.sh)
Thanks for your help Rusko. I think I may have pinned it down to just one (or two) assets. I have https://www.bbc.co.uk/news and https://www.bbc.co.uk/weather/2636935 which is our local weather. I tested by switching these both off last night, and it has been running overnight (9 hours) without problems!!
There must be something on the BBC pages - perhaps embedded videos, or?? which upsets things.
I have just now switched the news back on, and will update shortly.
Almost got this to be reproducible, which is a start. If I add BBC news page, things go fine for perhaps half an hour. Then the Pi power light goes mad, but the display continues as rotate as I want.
Add BBC weather as well, and after a while (not sure how long) display hangs with power light going mad.
I’ve worked in IT for all my life (though not at the PI level for decades, it is 50 years ago since I wrote assembler code!) my guess is that there is something on the BBC pages which give the PI a lot of work. Just one it can manage, but with both it can’t work out one page before it is asked to process the next (BBC page) and gets further and further behind?
I am working with a Pi Zero, and don’t know enough about it to trace processes (as I would try in windows using Task Manager).
I think I may have cracked this. Displaying BBC News for 10 secs causes problems. Extend it to 30 seconds and it behaves much better. My guess is that it takes more than ten seconds to process and render the BBC News page - it’s a complex page, and then comes up with the ‘cookies’ popup - and so Screenly starts processing the next page while still processing the BBC News page. That might explain why the power light is flashing like mad? Anyway, extending the display time seems to have cracked it.
I’m back here. I’ve really been struggling, and in the end upgraded from a Pi Zero to a 3B+. Thought this might solve any processing problems, but I seem to be back where I was.
I had only four assets, BBC News UK, BBC weather UK for our village, Screenly Clock, and a Google Slides web page, which allows me to easily change pages without accessing the Screenly Pi.
I can’t keep it going for more than an hour or two.
Might someone else try . . .
Screenly clock
And tell me whether you can keep Screenly going for more than an hour or two?
I’m beginning to look at other display software, but I do like Screenly OSE a) 'cos it’s free, and b) 'cos I like the Pi (and experimenting).
Thanks,
@RobertE
I’ve created a PR for the fixing of this with memorylimits to the viewer-service which prevents the pi from hanging/freezing which is mainly due to uzbl browser consuming a lot of resources and not doing good memory management as a process, thus we need to force a direct memory limit on the service…
If you need this asap (while we wait for rusko to review the changes I’ve made) you can simply edit your service files (if you know how to SSH into the Pi or using attached keyboard and mouse and pressing ctrl+alt+f1 to get to console) , then you just do the following:
Make this file /etc/systemd/system/screenly-viewer.service , look like the one below (only the added parts , i put asterisks on them, but obviously when you add the lines dont put the asterisks):
[Service]
WorkingDirectory=/home/pi/screenly
User=pi
*MemoryAccounting=yes*
*MemoryLimit=60%*
*KillMode=control-group*
Environment=DISPLAY=:0.0
and this file: /etc/systemd/system/screenly-celery.service , the same:
[Unit]
Description=Screenly celery worker
After=redis-server.service
[Service]
WorkingDirectory=/home/pi/screenly
User=pi
ExecStart=/usr/local/bin/celery worker -A server.celery -B -n worker@screenly --loglevel=info --schedule /home/pi/.screenly/celerybeat-schedule
Restart=always
RestartSec=5
*MemoryAccounting=yes*
*MemoryLimit=100M*
*KillMode=control-group*
Then restart and you should not have a freezing Pi anymore.
This basically restarts the screenly service anytime the assets you add reach / consume more than 60% of the Pi’s memory, we dont want this too small because it would cause the viewer to restart a bit of unnecessary times, while setting it too high will cause it to have no effect because by the time you consume 60% with just the screenly service, the other processes/services running would have consumed the rest and thus the Pi would still hang/freeze. I found this 60% to be best balance.
That’s fantastic ealmonte. Thanks so much for investigating. I have on occasion SSHed into a Pi, but am not really familiar with Linux etc., and find even simple text editing quite tricky, so I might just wait for this perhaps to hit an update and/or production? Is that likely to be weeks or months do you think? If it might be months, I might have a go at SSH.
Hmm… it shouldn’t take weeks, I am just waiting for @rusko to test/approve the changes, then you would just run a quick upgrade to reinstall screenly with the changes.
If you have access to the Pi with a keyboard, I can write here what you can type to get the changes you need from my fork of screenly-ose, which contain the changes made, the command just replaces the contents of the two service files in your Pi with the updates you want, without having to reinstall everything… I will put the URLs in bit.ly as well maybe that’ll help…
If you decide you want to do this, just get to the prompt in Pi, and type these:
Full URLs:
sudo sh -c 'curl -s https://raw.githubusercontent.com/ealmonte32/screenly-ose/patch-7/ansible/roles/screenly/files/screenly-viewer.service > /etc/systemd/system/screenly-viewer.service'
sudo sh -c 'curl -s https://raw.githubusercontent.com/ealmonte32/screenly-ose/patch-8/ansible/roles/screenly/files/screenly-celery.service > /etc/systemd/system/screenly-celery.service'
The above are shortened using bit.ly as follows:
for viewer : https://bit.ly/2uuuDXC
for celery: https://bit.ly/391yL0m
sudo sh -c 'curl -s https://bit.ly/2uuuDXC > /etc/systemd/system/screenly-viewer.service'
sudo sh -c 'curl -s https://bit.ly/391yL0m > /etc/systemd/system/screenly-celery.service'
Once again very helpful ealmonte32, thanks. I haven’t been able to get to the remote site, but may do today or tomorrow. (I have setup port forwarding for http to the screenly control panel, but not for SSH.) I’ll give it a go and let you know.
Whoops. I probably blundered. Got SSH, got to command prompt, and copied and pasted both short bit.ly lines. Powered down the Pi and repowered, and now it won’t boot. First time it got stuck on blue screen and down in bottom left corner u-dev.service (? I didn’t write it down stupidly, so might not be quite that) and after two more repowers, each time it sticks on rc-local.service. Any thoughts, or should I re-flash from a clean image? or can I do a 'sudo something to force it to ‘upgrade’ back to the latest standard image? Or . . . ?? Not urgent, as it is not a critical display.
This is why I need to work on allowing the Screenly-OSE to have a button that turns on and off the SSH option and displays the WAN IP, for users that know how to work with SSH and port forwarding, etc… for these cases… but yeah it’s on the to-do list… just very busy for that now.
Wait, you did copied this entire command:
sudo sh -c 'curl -s https://bit.ly/2uuuDXC > /etc/systemd/system/screenly-viewer.service'
into the command line prompt and pressed enter, and it asked you for your password, and then did the same for the other service, and then rebooted and you are getting a black screen? that’s weird, all the command does is literally override your screenly-viewer service with those additions…
Ok, since it’ll be quite hard to troubleshoot exactly what’s wrong, it might be best to run the installer again from command line, by the way, those commands and edits are for screenly-ose master/development branch, is that the one you were using? or production? I think you should re run the installer from prompt, then when it asks, select the development/master branch, and then when you at least have that back up and running, i can tell you how to easily edit those files, literally you can use the editor nano to go into those files, just add the lines I tell you, and then you should be good to go… up to you how you want to do it.
P.S. I also suggest you make sure the Pi is using memory of 192 instead of the default.
To do this from prompt:
sudo raspi-config
select Advanced Options
select Memory Split
type in 192
press tab to select Ok, then tab into the Finish, select that press enter, requires reboot.
This will just help overall with the Pi displaying websites that sometimes require more memory for the GPU.
Hi, think I’ve cracked it. Started again from a clean image, and ran your sudo . . bit.ly code, and then had a look (using nano) at the files screenly-viewer and screenly-celery, and they both contained html code. Title - bit.ly, and lots else that I didn’t note down I’m afraid.
So I did a bash <(curl etc . . . to do a fresh ‘upgrade’, and used nano to change the files manually, rebooted, and at least it has come to life. (surprised how easy nano is. I’ve had to use vi on Linux once or twice and found it a nightmare!)
I’ve added BBC News and BBC Weather back to my assets, and will let you know how things go.
MANY thanks for all your hard work on this.
Not good, I’m afraid. Display has been stuck showing BBC Weather for minutes now (should be 20 secs) but the ‘System Info’ screen still shows it scrolling through the various assets, sleeping, etc. Green light on Pi is solid on. I’ll try t reboot, but am not optimistic that we’ve sorted this problem yet.
Interesting detail from System Info.
Memory Total: 811 | Used: 662 | Free: 31 | Shared: 35 | Buff: 9 | Available: 62
Should this have triggered a reboot, if 662 used out of 811 available.
Happy to do more diagnosis if it helps?
Sorry just checking on this now, sorry about the bit.ly links, i thought that curl would redirect and actually get you the contents of the service files…
Did you select the development version of screenly when you ran the bash install script?
Ok so now that you actually changed it, yes you must not have something written correctly as I have shown on the PR I made that all the assets I added to test triggered the control-group restart of the viewer, so we needed to limit both the viewer and celery services…
Can you check these PR requests and tell me if your services both look like this: (notice the Restart=always instead of on-failure for viewer because it was a recent update I made because it was left out of previous PR)
https://github.com/Screenly/screenly-ose/pull/1304/files
and
https://github.com/Screenly/screenly-ose/pull/1305/files
This is how my screenly-viewer.service file looks:
This is how my celery.service file looks:
Thanks for persevering with me. I did (I think) go for the development version. The first question was did I want the ‘experimental’ version, and I said no. Then master/development? and I said yes. I have checked my two service files, and screenly-viewer was set to Restart=on-failure. I have changed to always, rebooted, and will see what happens. Thanks again.