Difference between revisions of "Kubke Lab:Research/ABR/Notebook/2013/11/05"

From OpenWetWare
Jump to: navigation, search
(Fabiana)
(fix raw html notebook nav)
 
(16 intermediate revisions by 2 users not shown)
Line 2: Line 2:
 
|-
 
|-
 
|style="background-color: #EEE"|[[Image:owwnotebook_icon.png|128px]]<span style="font-size:22px;"> Hearing development in barn owls</span>
 
|style="background-color: #EEE"|[[Image:owwnotebook_icon.png|128px]]<span style="font-size:22px;"> Hearing development in barn owls</span>
|style="background-color: #F2F2F2" align="center"|<html><img src="/images/9/94/Report.png" border="0" /></html> [[{{#sub:{{FULLPAGENAME}}|0|-11}}|Main project page]]<br />{{#if:{{#lnpreventry:{{FULLPAGENAME}}}}|<html><img src="/images/c/c3/Resultset_previous.png" border="0" /></html>[[{{#lnpreventry:{{FULLPAGENAME}}}}{{!}}Previous entry]]<html>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</html>}}{{#if:{{#lnnextentry:{{FULLPAGENAME}}}}|[[{{#lnnextentry:{{FULLPAGENAME}}}}{{!}}Next entry]]<html><img src="/images/5/5c/Resultset_next.png" border="0" /></html>}}
+
|style="background-color: #F2F2F2" align="center"|[[File:Report.png|frameless|link={{#sub:{{FULLPAGENAME}}|0|-11}}]][[{{#sub:{{FULLPAGENAME}}|0|-11}}|Main project page]]<br />{{#if:{{#lnpreventry:{{FULLPAGENAME}}}}|[[File:Resultset_previous.png|frameless|link={{#lnpreventry:{{FULLPAGENAME}}}}]][[{{#lnpreventry:{{FULLPAGENAME}}}}{{!}}Previous entry]]&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}}{{#if:{{#lnnextentry:{{FULLPAGENAME}}}}|[[{{#lnnextentry:{{FULLPAGENAME}}}}{{!}}Next entry]][[File:Resultset_next.png|frameless|link={{#lnnextentry:{{FULLPAGENAME}}}}]]}}
 
|-
 
|-
 
| colspan="2"|
 
| colspan="2"|
Line 11: Line 11:
  
 
=General Entries=
 
=General Entries=
* Insert content here.
+
* Skyped Andy vs Fabiana - trying to get an idea of the structure of the raw data files and make sure that the extracted text files are ok. Seems everything is ok, header log file contains all header info (albeit in a strange format). Different version files have different info, eg, some versions have the SD of the data others don't. Need to consider when extracting/analysing that the number of columns etc are going to vary from file to file.
  
 
=Personal Entries=
 
=Personal Entries=
 
==Fabiana==
 
==Fabiana==
 +
===Reading the binary files ===
 
*trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.  
 
*trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.  
 
Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from  
 
Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from  
 
abr.h and abr_4_mac.c
 
abr.h and abr_4_mac.c
 +
 +
According to andy need to
 +
Read the first 1128 bytes
 +
then read the next 1000 2-byte int
 +
skip 52
 +
read next 1000 points,
 +
and so on
 +
 +
<pre>
 +
rawData = file("189L0A.ABR", "rb")
 +
a <- readBin(rawData,integer(),file.info("189L0A.ABR")$size,size = 2)
 +
#size2 says integer is 2 bytes
 +
close(rawData)
 +
plot(a)
 +
plot(a, ylim = c(-500,500))
 +
#at the begining there is 564points that correspond to the header
 +
#If I clean up the header
 +
plot(a[562:2565], type="line")
 +
str(a)
 +
b<-seq(1,length(a), 2)
 +
c<-seq(2,length(a),2)
 +
plot(a[b])
 +
plot(a[c])
 +
aclean<-a[562:2565]
 +
   
 +
head(aclean)
 +
head(aclean[b])
 +
head(aclean[c])
 +
plot(aclean[b], xlim=c(0,1000))
 +
plot(aclean[c], xlim=c(0,1000))
 +
 +
 +
</pre>
 +
 +
=== Extracting info from log file===
 +
# From file 2013-11-050b-MFK.Rmd (in sandbox)
 +
 +
<html>
 +
 +
<body>
 +
<h1>Trying to extract info from the log files</h1>
 +
 +
<p>Filename: &ldquo;419L76.ABR.log&rdquo;</p>
 +
 +
<p>This is what the notepad shows:</p>
 +
 +
<p>Owl  Date  Time    Clock_kHz  n_reps  freq    version
 +
419 05/30/93    14:22:22    50  25  5000    5.30
 +
    Comment line: Luxman -43 Auxkl B norm 100 valium used
 +
Channel 0 data, only</p>
 +
 +
<p>delay
 +
0
 +
-160
 +
-100
 +
-180
 +
-140</p>
 +
 +
<p>I want to extract
 +
[1:7] -&gt; headers
 +
[8:14] -&gt; info
 +
rest -&gt; comment line
 +
tail -&gt; delays</p>
 +
 +
<pre><code class="r">logfile &lt;- read.delim2(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, header = T)
 +
head(logfile)
 +
</code></pre>
 +
 +
<pre><code>##                    Owl
 +
## 1                  419
 +
## 2                   
 +
## 3 Channel 0 data, only
 +
## 4                delay
 +
## 5                    0
 +
## 6                -160
 +
##                                                    Date    Time Clock_kHz
 +
## 1                                              05/30/93 14:22:22        50
 +
## 2 Comment line: Luxman -43 Auxkl B norm 100 valium used                NA
 +
## 3                                                                      NA
 +
## 4                                                                      NA
 +
## 5                                                                      NA
 +
## 6                                                                      NA
 +
##  n_reps freq version
 +
## 1    25 5000    5.30
 +
## 2    NA  NA       
 +
## 3    NA  NA       
 +
## 4    NA  NA       
 +
## 5    NA  NA       
 +
## 6    NA  NA
 +
</code></pre>
 +
 +
<pre><code class="r">str(logfile)
 +
</code></pre>
 +
 +
<pre><code>## &#39;data.frame&#39;:    9 obs. of  7 variables:
 +
##  $ Owl      : Factor w/ 9 levels &quot;&quot;,&quot;-100&quot;,&quot;-140&quot;,..: 7 1 8 9 6 4 2 5 3
 +
##  $ Date    : Factor w/ 3 levels &quot;&quot;,&quot;05/30/93&quot;,..: 2 3 1 1 1 1 1 1 1
 +
##  $ Time    : Factor w/ 2 levels &quot;&quot;,&quot;14:22:22&quot;: 2 1 1 1 1 1 1 1 1
 +
##  $ Clock_kHz: int  50 NA NA NA NA NA NA NA NA
 +
##  $ n_reps  : int  25 NA NA NA NA NA NA NA NA
 +
##  $ freq    : int  5000 NA NA NA NA NA NA NA NA
 +
##  $ version  : Factor w/ 2 levels &quot;&quot;,&quot; 5.30&quot;: 2 1 1 1 1 1 1 1 1
 +
</code></pre>
 +
 +
<p>I get a funny looking table</p>
 +
 +
<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0)
 +
</code></pre>
 +
 +
<p>This gets me the descriptor - still need to get the other rows</p>
 +
 +
<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0)
 +
loganaesth &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, nrows = 1, skip = 2)
 +
logtail &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, col.names = &quot;Owl&quot;, nrows = 20,
 +
    skip = 3)
 +
print(loghead)
 +
</code></pre>
 +
 +
<pre><code>##  Owl    Date    Time Clock_kHz n_reps freq version
 +
## 1 419 05/30/93 14:22:22        50    25 5000    5.3
 +
</code></pre>
 +
 +
<pre><code class="r">print(loganaesth)
 +
</code></pre>
 +
 +
<pre><code>##  V1                                                    V2
 +
## 1 NA Comment line: Luxman -43 Auxkl B norm 100 valium used
 +
</code></pre>
 +
 +
<pre><code class="r">print(logtail)
 +
</code></pre>
 +
 +
<pre><code>##                    Owl
 +
## 1 Channel 0 data, only
 +
## 2                delay
 +
## 3                    0
 +
## 4                -160
 +
## 5                -100
 +
## 6                -180
 +
## 7                -140
 +
</code></pre>
 +
 +
<p>Data file 419L76.ABR.txt
 +
read.table gives me an error that there is a mismatch betweeen columns and column names - but read.delim2 seems to solve the problem, although it creates a last column x with NAs (which is probably what is generating the problem in hte first place.)</p>
 +
 +
<pre><code class="r">data &lt;- read.delim2(&quot;419L76.ABR.txt&quot;, sep = &quot;\t&quot;, header = T)
 +
head(data)
 +
</code></pre>
 +
 +
<pre><code>##      msec left right spont bin.0 bin.1 bin.2 bin.3 bin.4 sdev_l sdev_r
 +
## 1    0.00  -93  -39    37  -49  -23    -1  -25  -140    101    107
 +
## 2    0.02    0  -49    20  -67  -12    86  158  -146    187    104
 +
## 3    0.04  -69  -70    21  -75  -23    24    3  -149    88    100
 +
## 4    0.06  -71  -107    35  -75  -38    35    4  -120    89    99
 +
## 5    0.08  -67  -117    16  -67  -45    41    8  -84    94    107
 +
## 6    0.10  -48  -98  -22  -46  -49    34    9  -71    95    103
 +
##  sdev_spont sd_bin.0 sd_bin.1 sd_bin.2 sd_bin.3 sd_bin.4  X
 +
## 1        112      86      101      129      126      116 NA
 +
## 2        112      101      96      195      260      118 NA
 +
## 3        109      113      100      123      128      110 NA
 +
## 4        106      108      106      130      119      99 NA
 +
## 5        99      95      106      140      117      99 NA
 +
## 6        95      91      109      139      115      104 NA
 +
</code></pre>
 +
 +
<p>returns 5 columns of bin.[0:4] - which probably correspond to the 5 delays in the log file. </p>
 +
 +
</body>
 +
 +
</html>
  
 
==Andy==
 
==Andy==
 +
 +
===ABR.H header file===
 +
 
Following file is from the ABR.H header file that was used for the original files.
 
Following file is from the ABR.H header file that was used for the original files.
  
File layout is probably: <Master Header><Sub Structure><1000 2-byte INT><Sub Structure><1000 2-byte INT> ...
+
File layout is probably: <Master_Header><Sub_Structure><1000 2-byte INT><Sub_Structure><1000 2-byte INT> ...
 +
 
 
This structure needs to be confirmed.
 
This structure needs to be confirmed.
  
 +
 +
----
  
 
typedef struct
 
typedef struct
Line 76: Line 253:
 
     short gain;
 
     short gain;
 
     char extra_room[1024-(12+12+12+12+80+2+2+1+2+2+2)];/* Add two more to correct CHAR */
 
     char extra_room[1024-(12+12+12+12+80+2+2+1+2+2+2)];/* Add two more to correct CHAR */
//    char extra_room[1024-(12+12+12+12+80+2+2+1)];/* Add two more to correct CHAR */
+
    //    char extra_room[1024-(12+12+12+12+80+2+2+1)];/* Add two more to correct CHAR */
 
     } HEADER_INFO,  EXPERIMENT_HEADER; // Bytes 1024
 
     } HEADER_INFO,  EXPERIMENT_HEADER; // Bytes 1024
  
Line 98: Line 275:
 
     AD_INFO input;      //    8
 
     AD_INFO input;      //    8
 
     } EXPERIMENT_INFO; // 52 bytes
 
     } EXPERIMENT_INFO; // 52 bytes
 +
 +
----
  
 
==Oris==
 
==Oris==

Latest revision as of 23:31, 26 September 2017

Owwnotebook icon.png Hearing development in barn owls Report.pngMain project page
Resultset previous.pngPrevious entry      Next entryResultset next.png


General Entries

  • Skyped Andy vs Fabiana - trying to get an idea of the structure of the raw data files and make sure that the extracted text files are ok. Seems everything is ok, header log file contains all header info (albeit in a strange format). Different version files have different info, eg, some versions have the SD of the data others don't. Need to consider when extracting/analysing that the number of columns etc are going to vary from file to file.

Personal Entries

Fabiana

Reading the binary files

  • trying to open the actual .ABR files - getting nowhere. Noticed the log files that were extracted dont have the info I need and need to see what info is in the actual data files.

Andy says it is an intel file - over skype trying to figure out the structure so we can read it - extracting info from abr.h and abr_4_mac.c

According to andy need to Read the first 1128 bytes then read the next 1000 2-byte int skip 52 read next 1000 points, and so on

rawData = file("189L0A.ABR", "rb")
a <- readBin(rawData,integer(),file.info("189L0A.ABR")$size,size = 2) 
#size2 says integer is 2 bytes
close(rawData)
plot(a) 
plot(a, ylim = c(-500,500))
#at the begining there is 564points that correspond to the header
#If I clean up the header 
plot(a[562:2565], type="line")
str(a)
b<-seq(1,length(a), 2)
c<-seq(2,length(a),2)
plot(a[b]) 
plot(a[c])
aclean<-a[562:2565]
     
head(aclean)
head(aclean[b])
head(aclean[c])
plot(aclean[b], xlim=c(0,1000))
plot(aclean[c], xlim=c(0,1000))


Extracting info from log file

  1. From file 2013-11-050b-MFK.Rmd (in sandbox)

<html>

<body> <h1>Trying to extract info from the log files</h1>

<p>Filename: &ldquo;419L76.ABR.log&rdquo;</p>

<p>This is what the notepad shows:</p>

<p>Owl Date Time Clock_kHz n_reps freq version 419 05/30/93 14:22:22 50 25 5000 5.30

   Comment line: Luxman -43 Auxkl B norm 100 valium used

Channel 0 data, only</p>

<p>delay 0 -160 -100 -180 -140</p>

<p>I want to extract [1:7] -&gt; headers [8:14] -&gt; info rest -&gt; comment line tail -&gt; delays</p>

<pre><code class="r">logfile &lt;- read.delim2(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, header = T) head(logfile) </code></pre>

<pre><code>## Owl

    1. 1 419
    2. 2
    3. 3 Channel 0 data, only
    4. 4 delay
    5. 5 0
    6. 6 -160
    7. Date Time Clock_kHz
    8. 1 05/30/93 14:22:22 50
    9. 2 Comment line: Luxman -43 Auxkl B norm 100 valium used NA
    10. 3 NA
    11. 4 NA
    12. 5 NA
    13. 6 NA
    14. n_reps freq version
    15. 1 25 5000 5.30
    16. 2 NA NA
    17. 3 NA NA
    18. 4 NA NA
    19. 5 NA NA
    20. 6 NA NA

</code></pre>

<pre><code class="r">str(logfile) </code></pre>

<pre><code>## &#39;data.frame&#39;: 9 obs. of 7 variables:

    1. $ Owl  : Factor w/ 9 levels &quot;&quot;,&quot;-100&quot;,&quot;-140&quot;,..: 7 1 8 9 6 4 2 5 3
    2. $ Date  : Factor w/ 3 levels &quot;&quot;,&quot;05/30/93&quot;,..: 2 3 1 1 1 1 1 1 1
    3. $ Time  : Factor w/ 2 levels &quot;&quot;,&quot;14:22:22&quot;: 2 1 1 1 1 1 1 1 1
    4. $ Clock_kHz: int 50 NA NA NA NA NA NA NA NA
    5. $ n_reps  : int 25 NA NA NA NA NA NA NA NA
    6. $ freq  : int 5000 NA NA NA NA NA NA NA NA
    7. $ version  : Factor w/ 2 levels &quot;&quot;,&quot; 5.30&quot;: 2 1 1 1 1 1 1 1 1

</code></pre>

<p>I get a funny looking table</p>

<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0) </code></pre>

<p>This gets me the descriptor - still need to get the other rows</p>

<pre><code class="r">loghead &lt;- read.table(&quot;419L76.ABR.log&quot;, header = T, sep = &quot;\t&quot;, nrows = 1, skip = 0) loganaesth &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, nrows = 1, skip = 2) logtail &lt;- read.table(&quot;419L76.ABR.log&quot;, sep = &quot;\t&quot;, col.names = &quot;Owl&quot;, nrows = 20,

   skip = 3)

print(loghead) </code></pre>

<pre><code>## Owl Date Time Clock_kHz n_reps freq version

    1. 1 419 05/30/93 14:22:22 50 25 5000 5.3

</code></pre>

<pre><code class="r">print(loganaesth) </code></pre>

<pre><code>## V1 V2

    1. 1 NA Comment line: Luxman -43 Auxkl B norm 100 valium used

</code></pre>

<pre><code class="r">print(logtail) </code></pre>

<pre><code>## Owl

    1. 1 Channel 0 data, only
    2. 2 delay
    3. 3 0
    4. 4 -160
    5. 5 -100
    6. 6 -180
    7. 7 -140

</code></pre>

<p>Data file 419L76.ABR.txt read.table gives me an error that there is a mismatch betweeen columns and column names - but read.delim2 seems to solve the problem, although it creates a last column x with NAs (which is probably what is generating the problem in hte first place.)</p>

<pre><code class="r">data &lt;- read.delim2(&quot;419L76.ABR.txt&quot;, sep = &quot;\t&quot;, header = T) head(data) </code></pre>

<pre><code>## msec left right spont bin.0 bin.1 bin.2 bin.3 bin.4 sdev_l sdev_r

    1. 1 0.00 -93 -39 37 -49 -23 -1 -25 -140 101 107
    2. 2 0.02 0 -49 20 -67 -12 86 158 -146 187 104
    3. 3 0.04 -69 -70 21 -75 -23 24 3 -149 88 100
    4. 4 0.06 -71 -107 35 -75 -38 35 4 -120 89 99
    5. 5 0.08 -67 -117 16 -67 -45 41 8 -84 94 107
    6. 6 0.10 -48 -98 -22 -46 -49 34 9 -71 95 103
    7. sdev_spont sd_bin.0 sd_bin.1 sd_bin.2 sd_bin.3 sd_bin.4 X
    8. 1 112 86 101 129 126 116 NA
    9. 2 112 101 96 195 260 118 NA
    10. 3 109 113 100 123 128 110 NA
    11. 4 106 108 106 130 119 99 NA
    12. 5 99 95 106 140 117 99 NA
    13. 6 95 91 109 139 115 104 NA

</code></pre>

<p>returns 5 columns of bin.[0:4] - which probably correspond to the 5 delays in the log file. </p>

</body>

</html>

Andy

ABR.H header file

Following file is from the ABR.H header file that was used for the original files.

File layout is probably: <Master_Header><Sub_Structure><1000 2-byte INT><Sub_Structure><1000 2-byte INT> ...

This structure needs to be confirmed.



typedef struct

   {
   short freq;	/* frequency in Hz */
   short amp;	/* amplitude in D/A units */
   short dur;		/* duration in clock ticks */
   short rise_fall;	/* rise_fall in clock ticks */
   } SIGNAL; // 8 bytes

typedef struct

   {
   SIGNAL carrier;
   SIGNAL envelope;
   } STIMULUS;

typedef struct

   {
   short left_delay;	/* microseconds to left  onset */
   short right_delay;	/* microseconds to right onset */
   short rep_shorterval;	/* interval in millseconds */
   } TIMING; //6 bytes

typedef struct

   {
   short clock;	    /* clock frequency in kHz */
   short max_size;   /* maximum size of each array */
   short channels;   /* 1 for carrier only, 2 for carrier & envelope */
   } DA_INFO; //6 bytes

typedef struct

   {
   short clock;	    /* in kHz */
   short size;	    /* number of elements (per channel) */
   short channels;   /* number of channels */
   short avg_reps;   /* number of reps making up the average */
   } AD_INFO; //8 bytes

typedef struct

   {
   char date[12];
   char time[12];
   char owl_id[12];
   char exp_id[12];
   char comments[80];
   short depth;
   short version;
   char side;
   short num_chan;
   short num_tests;
   short gain;
   char extra_room[1024-(12+12+12+12+80+2+2+1+2+2+2)];/* Add two more to correct CHAR */
   //    char extra_room[1024-(12+12+12+12+80+2+2+1)];/* Add two more to correct CHAR */
   } HEADER_INFO,  EXPERIMENT_HEADER; // Bytes 1024

typedef struct /**** THE MASTER STRUCTURE ****/

   {
   HEADER_INFO header; //1024
   STIMULUS left;      //  16
   STIMULUS right;     //  16
   TIMING delay;       //   6
   DA_INFO output;     //   6
   AD_INFO input;      //   8
   } EXPERIMENT;   // 1076 bytes


typedef struct /**** SUB STRUCTURE ****/

   {
   STIMULUS left;      //    16
   STIMULUS right;     //    16
   TIMING delay;       //     6
   DA_INFO output;     //     6
   AD_INFO input;      //     8
   } EXPERIMENT_INFO; // 52 bytes

Oris

  • Enter content here