The serial port gave up the ghost and I&...

English

So tonight I was going to hack on my API for realtime access to the bedside Zeo's data.  Instead I got a bunch of "Capture error: Read timeout in sync."

Now, the whole serial-over-FTDI-to-USB setup has always been a bit flaky.  The capture error I mentioned is a sign that the (physical) connection is broken between the Zeo's serial port (or circuits) and the serial-to-USB chip in the FTDI cable.  Usually this means I have to jiggle the thing a bit.  There's no latching mechanism, strain releif, or any other manner of hardening for the serial port, which tends to mean the mutant FTDI cable is starting to fall off or a pin was bent slightly away from the conductors in the serial connector.  Well, after bending pins a bit and tweaking for a while, I still get capture errors.  I also tried a second working FTDI cable I had laying around.  It makes me wonder if one of the solder connections on the Zeo's circuitboard has broken and the serial port is completely open and unconnected.  No more hacking on this tonight. 

This comes after some negative experiences already with the whole serial port thing.  For one, if anyone is to use the software I write, they are going to first have to do some time-consuming wiring in order to make a cable that plugs into the Zeo.  Far from plug-n-play, and the situation doesn't look like it is changing.  Now, making a durable Zeo-to-computer cable is tricky.  I've gone through a few already.  Then I have to mess with all of that connector crimping and forming all over again each time.  So far I've settled on using epoxy to freeze the little wires spliced out of the FTDI cable and running into the serial connector.  Be sure to cover a large amount of the unspliced cable with the epoxy, or it will break loose and the strain releif is no good anymore.  Maybe sand/gouge the unspliced cable sleeve a bit to make the epoxy grab better.  My first attempt was with hot glue, and that lasted a very short time (<1 month) before failing.  But that's solvable at least, mostly.  There's still no (easy) way to make sure the connector actually stays attached to the Zeo.  I think my latest attempt involved carefully sanding the some tab on the connector until it was just right, which gave it a small amount of snap due to friction alone.  This thing is also in a very inconvenient spot on the Zeo: the bottom.  This requires the cable I make to be small and at a right angle, lest the Zeo rest ON the cable and cause all kinds of connectivity havoc (because it makes the serial port pins bend and move around inside the connector).

So I'm going to call it "not worth fixing" and give up for now. 

I'd probably stick with it if there were a USB port on this thing.  Or maybe if Zeo sold a higher quality Zeo-to-PC cable, though I suspect it'd be too late for my Zeo. 

Don't get me wrong, I'm really glad you guys opened this stuff up and let me do some hacking to begin with.  That was really important a little over a year ago!  Thanks!

The Zeo Sleep Manager seems like the way to go anyways.  I have an android phone, so I might buy one.  I may even try the android API, though I'll have to feel like dealing with JavaAndItsAmazinglyVerboseGrammar before I slog through it.  (C API would be cool.)

Forums [fr:field:taxonomy_forums:forum:label]:

I've got the same issue - I asked about it a few weeks ago in this post. It's disappointing to hear that it's a hardware issue, as I'm really not inclined to buy another cable after the last one never worked. I'm considering writing some code to re-connect, as it does seem capable of recovering, but it would leave gaps in the data and would be a pain in general. All in all, I appreciate that Zeo has opened up the hardware and the API, but like you say, it's more hassle than it's worth. As it stands now, my Zeo isn't very useful, as most of my "awake" moments are recorded as "deep sleep" - I really wanted to be able to investigate the raw data and try to fix the problem.

I'm considering the Mobile version as well, though I'm a bit worried that I'd just be pouring money into a problem that doesn't seem to work in my new apartment. Zeo's definitely helped me focus on sleep over the past few months, I just wish I could get more realistic results! I wonder if I could trade in my Bedside for a Mobile...

By the way, there is a C API that was made by TJ on these forums: http://www.ecstaticlyrics.com/secret/zeo_raw_data_parser.tar.bz2 (not for Android though).

I just noticed that your code is written in C as well. Sorry, I got confused by your "C API would be cool" statement - I guess you meant for Android.

I think the best strain relief is none at all.

Most of the time when you're talking about strain relief, you're talking about connectors that connect firmly and securely in place (like the old screw-on serial port connectors) where you then decide you're going to pick up the device and move it around constantly, but you don't want the cable to get broken in half near the connector since that is where it undergoes the most strain.  Another good example is coaxial cable attached to a television.  That will come apart easily if you move the television around a lot.  However, you're also likely to bend the connector on the television, as it isn't designed to provide strain relief.

...and that's where I think your problem is.  The Zeo isn't designed to provide strain relief either.  That's just a header pin connector in there.  As you say, the connector doesn't even snap into place.  The hole wasn't designed for any connector (despite claims to the opposite, just take it apart and look at it, it's obviously just a hole).  Thus, to expect to build up a cable and connector with epoxy and have strain forces absorbed by the Zeo instead of the cable itself is only going to lead to a broken Zeo.  

The best place to focus your strain forces is into the wire itself.  It's the easiest component to replace.  Just strip of a little more insulation and attach a new connector.  If you buy solder-type header pin connectors instead of the crimp-on connectors, it'll cost you a dime every time you have to do it.  ...and if you do it right, you won't be doing it often.

Just strip off some of the outer black insulation from the FTDI cable.  (I just rely on what's already exposed, but if you ever have to repair it, you may need to strip some off.)  This way, all of the strain is easily delt with by the three thin flexible wires which go from the bulky black part of the FTDI cable to the header pin connector in the Zeo.  Now when you move the Zeo around, just those three little wires have to bend, and they bend rather easily, so it's no big deal.  I've been using my Zeo this way for seven months now and I've had no problems whatsoever.  The cable hasn't even accidentally come loose, though I make sure to check that it's still fully inserted every time I move the Zeo (which is rare so it's no big deal).

As for the Zeo having a USB connector, I agree it'd be nice, but I understand why it doesn't.  After moving to a new house two weeks ago, I've been living in an arrangement where my bed simply isn't anywhere near my computer.  So when I want to view my sleep data, I take the SD card out of the Zeo and load the data into my computer.  It's quite easy to do and is a very simple solution to recording data without having to sleep in the same room with a noisy computer turned on all night.  So I totally understand why it works that way.

I'd like to see the raw data written to the SD card as well.  (Also fix all the bugs, they're quite annoying, particularly the 16 hour limit on sleep length.)  Having the option for live data is nice, but for simply doing your own analysis on the waveform data, just reading it from the card in the morning is adequate, and that would make it rather easy for people to use the random applications people make.  I only wanted the live data initially so that it was easy to sync the Zeo's data with other data I collect, since sometimes I like to record respiration data and trying to sync it up with the Zeo data in the morning would be difficult.  It's much easier to just record both data streams simultaneously, then there's no question of how they sync together.

I am surprised that you're still working on that library.  I thought the piece of code I put together for you demonstrated quite nicely that the protocol, while more complicated than necessary, just isn't all that difficult to deal with.  Honestly, I don't see a need for a library at all, other than that Zeo hasn't provided much in the way of documentation.  For the live data I mostly went by the quick summary that is in one of the source files for the library, then referred to the code itself (though it is written in a language I don't understand) to figure out a few details.  For the SD card data, I just used the data dumping utility to find what data I was looking for, then looked at hex dumps of the binary data to figure out where it was stored.  That might take a while if I cared to find all of the data, but after finding the hypnogram data and the start times, I decided I didn't really care about any of the other data.  

I think everyone is focusing too much on the live data stream anyway.  For the most part, all I've used it for is to make up for the bug in the SD card data where, when the Zeo decides to join two sleep periods (you wake up for an hour, then go back to sleep) it fills in the gap in the hypnogram data with code 1 (awake) instead of code 0 (no data) which it should use since, as the headband wasn't in use, it doesn't really know that I was awake.  (plus I like to see "awake but trying to sleep" show up differently than "awake and not even in bed" in my reports)  Using the live data I can properly record that the headband wasn't in use at the time.  While I've been recording the raw data for months, the only portion I've been using is the sleep stage data, which I could just as easily read from the SD card were it not for that bug. 

So I've been making reports like these: 

http://www.ecstaticlyrics.com/secret/seven_months.pdf

Color codes:  black = REM, dark blue = deep, light blue = light, yellow = awake, white = not in bed / unknown.  At the end, you see the lack of yellow...  That's because I just discard the awake codes from the SD card since so many of them should actually be no data codes which are drawn in white. 

Using this, I've discovered a strange link between caffeine use and interruptions in my REM sleep.  Even small amounts of caffeine, like a can of soda in the morning, will affect my sleep that night, and larger amounts will affect the next night as well.  However, if I cease all consumption of caffeine, there's an improvement for a few days, then it goes back to how it was before.  So lately I've been eating a chocolate bar in the morning for the even smaller amount of caffeine it contains, to see if I can keep myself in that sweet spot where my REM sleep benefits.  So far it seems to be working, but it really needs a lot more study to reach a conclusion.  In particular, I'm always trying multiple things at once, so I need to see it work for like six months to really believe it.

It seems to me that anyone who wants to make a truly useful application should probably forget about the raw waveform data.  It's difficult to interpret, and so I haven't seen anyone try.  Instead they just display the wiggly lines as if that will do anything but amuse people.

I've thought about rewriting my PDF generating application so it's useable by other people.  It's a mess of Perl scripts and command line utilitys (and for Linux at that) at the moment, so I can't really offer it to the general public.  However, I'm generally too tired to do such complex things.  There's a reason it's such a crude hack: that's all the energy I had when I wrote it.

I think people could do a lot of good to write some applications that work on the raw waveform data.  One idea would be to have an application which you can program with any supplements you might be taking and their half-lives, so that you can tell the program when and how much you take, and it can calculate the exact amount in your bloodstream from moment to moment.  Then it can determine if the supplement is having any effect on your sleep, and also determine what the ideal amount is.  Thus, if I programmed it for caffeine, and told it every time I consumed some soda or chocolate or coffee, it'd be able to tell me that my ideal amount of caffeine is 5 mg or whatever it is.  (I'm still rather surprised that the ideal amount isn't zero.)  I've also been trying other supplements like Choline, which the body uses to make acetylcholine, which is the main neurotransmitter used for throat muscles I beleive, and so may help to keep my airway open.  However, both too much and too little acetylcholine will weaken the muscles, so it's definately the sort of thing where finding exactly the right dose would be necessary.

Personally, I'm just amused with the way my data is displayed in that PDF.  It's really far more workable than anything that attempts to display sleep as "nights" since my sleep is far too irregular for that.  So even if all your application offered was a different way to view the same data, it might just be exactly what some other people are looking for.  ...and if all you're doing is reading the SD card data, no one has to hack together a USB cable to use your application.  They can just run it on the data they already copy to their computer all the time anyway.  (Well, they still have to install the firmware update, I guess.)

Correction:  I think people could do a lot of good to write some applications that work on the data from the SD card.

Bascially, if the cable refuses to work for you, you can probably find pleanty of useful and interesting things to do without it.  I've been recording that data on my disk for months (3.3 GB so far) but so far I have no idead what to do with it, and so I haven't bothered to set up recording of it again since I've moved.  Seems a little pointless since most everything I really care about is on the SD card anyway.  So work on an application with better ways to track what affects your sleep, or build a web site that lets people upload their SD card data and share it with anyone who wants to download it.  (I'd love to get other people's data myself, in order to analyse in what specific ways my sleep differs from other people's, and also maybe to see how many other people seem to have sleep disturbances similar to my own.)  I really think the raw data is distracting everyone from a lot of cool things that could be done with only what is on the SD card.

...

 

As for strain relief:  I don't like to allow little wires to bend at sharp angles because when metal is bent back and forth repeatedly it creates fissures that grow and eventually break the wire apart.  It makes sense though that placing this strain on the connection inside the Zeo is a bad idea, since that will suffer the same fate and is much harder to repair.  As much as this is annoying for the FTDI connection, I'm pretty sure the power connection failing is what ghosted my Zeo.  I wouldn't be surprised if some solder connection broke on the power connector due to repeatedly (un)plugging it and moving it around.  I don't feel like taking the time to take it apart though.

 

Btw, I did buy the Sleep Manager and have been using on my droid phone for almost the entire time since the original post.  It is way more convenient.  It is kinda too bad though to be back to this state where I can't look at 30-second granularity hypnograms after napping (back to 5 minute granularity like unmodded bedside unit) and the thing only seems to acknowledge one sleep session per day.  That one session per day thing is a problem for people like me that have about 5.  Still, even with that shortcoming, it is probably more useful than the modded bedside unit for me right now: i'm not really recording data at this point, but instead using feedback from the Zeo to plan my sleep, and that doesn't require me to track all sessions per day. 

 

The SD card was not that useful for me earlier on.  It is inconvenient for quickly glancing at how much REM I got, and it doesn't allow me to do nifty things like trigger alarms based on a large patch of stale light sleep (realtime is useful!). 

 

I think the manager has great potential if the right code is supplied for it.  I mean, the phone IS a computer, so the physical integration becomes trivial.  No need to worry about if there is a computer nearby or not.  And since the headband can last on it's own batteries for a while, and the phone can last on it's own batteries for a while, no 110V supply is needed!  That it happens to be in not just any computer, but the phone in particular, is also an opportunity: in principle it can prevent my phone from ringing when I am asleep, and then turn the ringer back on when I'm awake.  Then with some more polyphasic-specific features I'll be very happy.  Even if someone is considering the manager but lacks a smartphone, I think it would be worth it to buy a cheap $50 droid phone off of Ebay and run it with no cell service.  I know Samsung Intercept would probably work for this.  It's a PoS, but serviceable and more than enough for this.  Then you have a super-mobile $150 sleep recording system.

 

Note that I was gathering raw-brainwave data for mostly archival purposes.  What if someone wanted to analyze my data in much greater detail later on?  With the raw data, they'd have everything the sensors could produce.  I'll admit that I've been pretty bad at actually putting this data out there and at doing more basic analyses in the meantime.  Even with some more time in my life, it tends to get prioritized to other tasks. 

 

Interesting info about that caffeine thing!  Strange, indeed.

 

 

 

I think the main advantage to the cable interface (with the same functionality provided by the Zeo mobile interface), is the real-time streaming. One thing I'd like to have it do is calculate ZQ on the fly so that I can glance at it and decide if I should suck it up and get up, or if I really do need the extra sleep. I'm terrible at assessing my sleep first thing in the morning - I've got some days that I gave a 1 for "morning feel" and then a 10 for "day feel", so if I could have some external guidance on that front I'd at least give it a shot.

Then there's the laziness aspect of not having to physically remove the card...

P.S. why is it that posts on this forum have to be queued for moderator approval and then take weeks to appear? I posted about this and other forum issues in the support forum, and then felt like an idiot when it posted immediately.