Locked up NanosG20 only boots after removing RTC battery

I just retrieved a NanosG20 from a remote location, which had become unresponsive and found that it was not running, and I could not boot it by pressing the button or resetting the power, until I physically removed the RTC battery. Having looked at the logs it looks like it had just woke up from mem sleep, and reconnected a USB modem. After that there is nothing.

I suspect this could be some kind of power issue, but I don't understand how it can have got the board into a state where it would not run again with a power cycle.

How can I ensure the board will not become locked up like this? I am considering using the watchdog facility, but I will need to pause this when I go into mem sleep. Would this prevent this weird power lockup?

I noticed this page, but cannot work out where J5 is on my board, and how this relates to my problem. http://armbedded.eu/node/354

Re: Locked up NanosG20 only boots after removing RTC battery

How long was the device in service before this error occurred?
It might be that the problem is caused by an empty or low battery.

Further explanation of the power-up issue can be found here:
http://www.ledato.de/download/NanosG20_rev2_description.pdf

J5 is a solder bridge (or a 0 Ohms resistor) between X3 and J3 near to the male DSUB9 connector.

Re: Locked up NanosG20 only boots after removing RTC battery

Thanks for the reply.

It was after only 10 days of autonomous operation before the device became unusable. The RTC battery is certainly not the problem as it was nearly brand new and should last for years. Removing it and replacing fixed the problem.

J5 already has 0 Ohm resistor- what does removing/adding this actually do? It's not clear from the decoration. Should I try removing it?

I need the device to always boot when power is applied, and I can't have it hang after going into mem sleep :-(

Re: Locked up NanosG20 only boots after removing RTC battery

J5 overrides the shutdown controller by pulling the enable pin of the 5V voltage regulator to GND, therefore removing it would only add further possibilities that the system won't boot. It is still a bit strange, that removing the battery solved the problem, because its influence on the boot is removed with J5 being soldered.

It is a long shot but one further question, do you use the power supply available at ledato.de for the NanosG20 or a different one? Maybe it doesn't cope well with the wake up situation.

Apart from that, it may just be a faulty unit. Have you access to another unit to test if it behaves the same? And maybe you are able to capture the last messages on the serial console before the device locks up further diagnostics.

Re: Locked up NanosG20 only boots after removing RTC battery

My colleague also has the same problem. I will try running the sleep/wake code really fast with a counter to see if it's an issue after a number of cycles. We have three boards between us, so I can test them.

It may be due to the power supply as there was another device using power and there is a modem connected, which may be drawing too much power when in areas of weak signal. The USB is switched off before going into mem sleep though, so on wakeup there should have been no power draw from it. Does the USB power just get passed straight through from the supply? Is there any limit on the power draw from the USB sockets?

For our next iteration we are going to provide a 1A power supply which supplies only the NanosG20 and no other peripherals.

To answer your question about logs, there was no output from dmesg or syslog immediately before the crash so there is no way to tell what happened unfortunately. My assumption was that it never woke up as I couldn't see usual messages about the device waking back up.

Re: Locked up NanosG20 only boots after removing RTC battery

Further to my last message, I was able to reproduce this problem in the office. It locked up after 917 loops of sleeping to memory.

This is the output from the serial debug console. The extra number in there is so that I can count how many times the loop has executed:

Restarting tasks ... done.
 916
PM: Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.01 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Suspending console(s) (use no_console_suspend to debug)
PM: suspend of devices complete after 1.991 msecs
PM: late suspend of devices complete after 0.329 msecs
PM: noirq suspend of devices complete after 0.309 msecs
PM: noirq resume of devices complete after 0.215 msecs
PM: early resume of devices complete after 0.208 msecs
usb usb1: root hub lost power or was reset
PM: resume of devices complete after 298.731 msecs
Restarting tasks ... done.
 917
PM: Syncing filesystems ... done.
Freezing user space processes ... (elapsed 0.01 seconds) done.
Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.
Suspending console(s) (use no_console_suspend to debug)

Re: Locked up NanosG20 only boots after removing RTC battery

I have just tested the additional board which locked up after 196 sleep/wake cycles. Here is the bash script should you wish to test it yourself:

#!/bin/bash

COUNTER=1

while true
do
    
    echo "" > /sys/class/rtc/rtc0/wakealarm
    echo "+4" > /sys/class/rtc/rtc0/wakealarm
    echo mem > /sys/power/state

    sleep 4s

    echo " $COUNTER "
    COUNTER=$[$COUNTER +1]

done

Re: Locked up NanosG20 only boots after removing RTC battery

Maybe the sleep time of 4 seconds is just too short. It may be that sometimes the system isn't fully suspended when the timer triggers and when the system is suspended there is no longer any alarm set. Do you also use such a short time in your final product?

Re: Locked up NanosG20 only boots after removing RTC battery

Hi,

This is certainly not the case. I am only running these tests due to the alarm failing in the first place. In production we do mem sleep for either 10 minutes, 1 hour, or 1 day- so certainly not enough time for this to be a problem. I have now encountered this problem twice in production on 3 different boards.

I settled on 4 seconds for the test after adding significant margin to ensure the device was sleeping well before the alarm triggered. Because the mem sleep tells you how long it took, and because I was watching it, I can confirm that the alarm either fails to wake the board, or fails to trigger- not that the board isn't in mem sleep. I could increase this delay to prove it, but I would rather you took my word for it.

At this stage I need to figure out;

  • Is there a pin on X11/X12/X8/J2/J9/J10 that I can use to wake the board from an external source, such as a reliable RTC device? I am planning on using a periodic alarm on a DS3231. I couldn't find any pins for attaching anything to the switch SW1, which does wake the board up.
  • Is it possible to use the Periodic Interval Timer mentioned in the technical specification to have a regular wakeup? If so, how do I do this?
  • Can the RTC be set on a periodic alarm such that if one is missed, the next alarm will wake it up? If yes, how?

Re: Locked up NanosG20 only boots after removing RTC battery

> Is there a pin on X11/X12/X8/J2/J9/J10 that I can use to wake the board from an external source, such as a reliable RTC device? I am planning on using a periodic alarm on a DS3231. I couldn't find any pins for attaching anything to the switch SW1, which does wake the board up.

In principle, you can use any IO pin as a wakeup source, but you have to adjust the NanosG20 board file in the kernel to declare this pin. If you use the standard kernel, use https://www.ledato.de/download/nanosg20.patch as a base, or if you use Debian Wheezy, use http://download.armbedded.eu/support/nanosg20/nanosg20-3.4.112.patch as the base patch. Just look for the GPIO button array and add your pin in the same way.

> Is it possible to use the Periodic Interval Timer mentioned in the technical specification to have a regular wakeup? If so, how do I do this?
I don't think this is possible. It is used for the kernel scheduler and should not be reprogrammed during sleep. It is also designed for much higher interrupt frequencies than you need.

> Can the RTC be set on a periodic alarm such that if one is missed, the next alarm will wake it up? If yes, how?
No, the RTC alarm can only be set to a single point in time.

Maybe you can get a periodic wake up interrupt with the Timer Counter Block of the AT91SAM9G20, but you would most probably have to add some C code to the kernel to configure them.

Syndicate content