[obdgpslogger] sampling rate

[ken f]

On 04/30/2014 11:34 PM Gary Briggs wrote:
> On Wed, Apr 30, 2014 at 10:21:24AM -0400, ken f wrote:
>> On 04/28/2014 12:14 PM Gary Briggs wrote:
>>> On Mon, Apr 28, 2014 at 05:10:02AM -0400, ken f wrote:
>>>> obdgpslogger is compiled and installed on my Raspberry Pi.  Initial
>>>> testing indicates it's working fine except the sampling rate is
>>>> significantly slower than what I specified, both on the command line
>>>> and in the config file.  In one test I used:
>>>>
>>>> obdgpslogger -b 38400 -d ~/logs/obddb-${time} -u
>>>> ~/logs/obdgpslogger-${time}.out -l ~/logs/obdgpslogger-${time}.log
>>>> -a 1 -i temp,iat,rpm,throttlepos,vss,fuelsys,load_pct
>>>>
>>>> (Ignore any line-wrapping your email client might have done.
>>>> Commands were entered all on a single line.)
>>>>
>>>> Instead of getting ECU data once per second (as per "-a 1"), it came
>>>> very close to once every three seconds.  I ran tests specifying in
>>>> the config file
>>>>
>>>> baudrate_upgrade=0
>>>>
>>>> and then
>>>>
>>>> optimisations=1
>>>>
>>>> but still only got back ECU data once per three seconds.
>>>>
>>>> During these tests I ran "top" and it showed a load average of only
>>>> 0.5 (thereabouts), so the code isn't at all taxing the Pi's
>>>> processor.
>>
>> I forgot to mention in my previous post that I also ran iotop and it
>> showed nothing which could be called a burden on I/O.
>
> Yeah, there's no actual traffic, but OBDGPSLogger blocks on serial reads
> [and writes]

Hmmm.  One guy I read said his code did reads and writes on the port in 
non-blocking mode, so that's what  I tried to do.  (Yeah, for a time I 
tried writing my own code for obd.


>
> The actual throughput is almost nil, but it's common to be waiting on
> the dongle, which is waiting on the car.
>
>>>>....
>>>>
>>>> So unless I'm missing something, I can only imagine that my car's
>>>> ECU is the bottleneck, and/or that it can't return the amount of
>>>> data I'm asking for any faster than once per three seconds.  (If it
>>>> matters, the car in question is a '98 BMW 318i.)
>>>>
>>>> What kind of speed are others getting?  Is anyone getting a better
>>>> rate than I am?  If so, how?
>>>>
>>>> ....
>>
>> Gary, if points of possible improvement were noted in the code
>> and/or in TODO, perhaps enterprising coders might take a run at
>> them.
>
> Heh. The biggest one is the db layout. I first started it before I knew
> much about DB design, so each sample type is a column, and the time is
> fixed for a row. Bad idea; much better would be time-PID-value 3-tuples,
> so samplerate can vary with different PIDs, and the schema cannot vary.

This would seem to amount to a separate table for each PID... or?


> ....
>>>
>>> OK, my immediate question is about hardware: What dongle do you have? Is
>>> it something cheap off ebay, or is it one of the premium ones?
>>
>> Good question.  The one I got is <http://www.obdtester.com/elm-usb>,
>> which claims not to be made in China, but other than that I have no
>> way of knowing or judging, not a clue about possible metrics or
>> anything else which might indicate its actual quality... in terms of
>> demonstrated functionality.  A little utility which would exercise
>> various dongles and produce useful metrics about them would be handy
>> here... and in a way totally subversive to those whose business
>> relies on ignorant consumers like myself.
>>
>> Are there any particular dongles known to perform well...? or some
>> already existing way to assess them objectively?
>
> I'm not familiar with that vendor at all. They may be new, there's a
> lot of new folks in the market right now. The only other thing I'd note
> [without attempting to pass judgement] is this: Their FAQ says 'most
> dongles are RS232 not USB, thus we're better'; that's a ... fairly dated
> idea. USB has been the standard for dongles for a few years, bluetooth
> has been the higher end option for three or four years, and wifi dongles
> are finally coming of age thanks to Apple insisting on crippling their
> bluetooth stack *grumble*
>
> I'm not sure of anyone who's really done objective tests and posted
> them online.
>
> Personally, I really love this vendor: http://www.scantool.net/
> Their dongles have a very high samplerate; there's a couple tiers of
> dongles; the higher tier ones are typically seeing sixty, seventy, maybe
> a hundred samples a second in default settings. The lower tier ones see
> five, ten, or twenty.
>
> What scantool.net have really going for them is their power management;
> the chip in their devices is the STN1110 [which they designed themselves],
> and it has the best capabilities around. If you don't mind the cabling,
> the SX draws power from USB rather than the car, so when the computer's
> off, there's literally no draw.
>
> [Full disclosure, scantool, among other vendors, sprang me free samples
> of their hardware; I paid for a bunch of ebay knockoffs, an OBDKey,
> and made a couple myself with arduino. My real dongle of choice is the
> Bluetooth OBDLink MX, the chip inside it is the same as the rest of
> scantool.net's range, though, so unless you specifically love the form
> factor and the bluetooth, the SX is the way to go]
>
>>>
>>> I've had experiences consistent with what you describe with cheapo
>>> ebay dongles. Low CPU utilisation could be that obdgpslogger is
>>> voluntarily sleeping, or it could mostly be blocked on serial i/o.
>>>
>>> To test it out, do this: set samplerate to zero ["-a 0"], and sample
>>> exactly one thing ["-i rpm"].
>>>
>>> In the output logfile, the number of rows is the raw number of samples
>>> taken [when you sample multiple PIDs, it'll be num_rows * num_PIDs]. So
>>> start there, do one run for a while, and see what observed samplerate
>>> you get. My guess is that it'll be about the same as you see right now.
>>
>> Just so I get this right the first time, that would be:
>>
>> obdgpslogger -s /dev/ttyACM0 -a 0 -i rpm -d dongle-rate.db &
>
> Yup
>
>> I guess I should also mv /etc/obdgpslogger off to something else to
>> prevent it from possibly confounding this dongle-rate test. (?)
>
> In good unix fashion [ http://www.catb.org/esr/writings/taoup/html/ch10s06.html ]
> obdgpslogger will honor command-line options to override whatever it
> finds in the config files.
>
>>> Another comparison to try is to run obdsim with the obdgpslogger you have,
>>> on the hardware platform you're currently using [pi + SDcard/etc]. With
>>> ELM optimisations enabled, obdsim generally is *fast* at responding.
>>> Give that a try, and see if your samplerate goes up; it's an
>>> easy way to identify if you have car-and-OBD-hardware-slowness or
>>> obdgpslogger-and-raspberrypi-slowness
>>
>> I'd love to do this one, but after reading the odbsim manpage back
>> and forth a few times, I still can't figure out what all the options
>> should be.  And, if obsim simulates a dongle, I suppose I should
>> then run the above obdgpslogger command, but instead of
>> /dev/ttyACM0, using the pseudo-device created by obdsim, yes?
>
> Yup!
>
> You also should be able to edit the obdgpslogger config file to taste,
> then cheat by running "obdsim -o", which launches obdsim and it'll
> automatically launch obdgpslogger and connect it to the pty.

Here's my results:

Using obdsim returned rpm data first at a rate above 200 times per 
second.  Then it slowed to between 170 and 200.  For a brief moment the 
rate dropped to 35 per second, then immediately resumed at 150 - 170. 
I'm guessing the Pi got busy with something momentarily, causing the 
drop to 35 per second.

Reading the rpms through the dongle got me a persistent rate of once 
every 0.42 seconds, quite a ways slower than obdsim's.

So the next question: is the slower rate from the dongle due to the 
dongle being slow or is that just the rate a '98 BMW's ECU outputs? 
Said another way, would some other dongle give me faster data collection?


>
>>>
>>> Your car pre-dates 2002, so some of the "optimisations" may not actually
>>> work properly with your car. That being said, my experience is that BMW
>>> are relatively well-behaved in that regard.
>>
>> Good to hear.  Yeah, BMW does a lot of stuff right and better than
>> the average car.  Makes it worth the higher entry and maintenance
>> costs... most of the time.
>
> Heh. I love my mini when it's working. But every time I take it to be
> worked on, it costs me a grand and I get insulted by too-highly-paid
> customer reps. Good times

Ditto here.  So I try to do repairs and maintenance myself when at all 
possible.  Compared to other makes of cars I've worked on, BMW seems to 
have thought through how their vehicles would be repaired and designed 
them to not be absurdly difficult.

This brings up another point: it seems that obdgpslogger doesn't read 
DTCs or clear the MIL or reset the oil change indicator.  Or did I miss 
these somehow?