Tessa A. Morris Electronic Lab Notebook

From OpenWetWare
Revision as of 16:52, 2 June 2015 by Tessa A. Morris (talk | contribs) (delete titles that are unnecessary)
Jump to navigationJump to search

Microarray Data Analysis Done (May 18- 26, 2015)

For the first week and a half of SURP 2015 the class all went through the process of Microarray data analysis. The methods and observations are recorded here.

16 Test Files from Dahlquist-data (May 26- 28, 2015)

GRNmap Testing Report 16 Test Files from Dahlquist-data 2015-05-26 TM

16 Test Files (June 1 - 2, 2015)

GRNmap Testing Page

GRNmap Testing Report 16 Test Files 2015-06-01

Selecting the GRNmap Version and Base Input Sheet

  1. Download the GRNmap-beta branch version from Github by downloading the zip file.
  2. Move it to the desktop and unzip by right-clicking on it and choosing 7-zip > Extract here.
  3. Open the unzipped "GRNmap-beta branch" folder and find the folder "test_files" then navigate to "estimation_tests" and open the file "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls"
    • We compared "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls" and "4-genes_6-edges_artificial-data_Sigmoid_forward.xlsx" and found them to contain different data points and different time points, so the "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls" was chosen as the base for the 16 test files.

Editing the Optimization Parameters for each Input

  • There are sixteen different input sheets that need to be tested. The "optimization parameters" worksheet is adjusted for each type. For all inputs, the following optimization parameters will be left as they were originally entered
    • "alpha" should be 1.00E-10
    • "kk_max" should be 1
    • "MaxIter" should be 1e08
    • "TolFun" should be 1e-10
    • "MaxFunEval" should be 1e08
    • "TolX" should be 1e-10
    • For the parameter "time" (Cell A13), should have "0.4", "0.8", "1.2", and "1.4"
    • For the parameter "Strain" (Cell A14), make sure it says "dcin5", making sure that the capitalization and spelling is the same as the worksheet containing that strain's expression data.
    • For the parameter "Sheet" (Cell A15), give the number of the worksheet from left to right that your "Strain" log2 expression data is in. This should be the fourth sheet.
    • For the parameter "Deletion", leave the zero in cell B15 (corresponding to wt). In cell C15, put a number corresponding to the position in the list of gene names that the gene that was deleted appears. This should be the number three in the list (disregard the column header in this count and only consider the actual gene names themselves).
    • For the parameter, "simtime", the row should read: 0, 0.1, 0.2, 0.3 <...fill by steps of 0.1...>, 2, each number in a different cell.
  • The parameters "Sigmoid","estimateParams", "makeGraphs","fix_P", and "fix_b" will be different for each input sheet.
  1. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 0
    • "fix_b" should be 0
  2. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_no-graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 0
    • "fix_b" should be 0
  3. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-1_graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 1
    • "fix_b" should be 0
  4. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-1_no-graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 1
    • "fix_b" should be 0
  5. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-0_graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 0
    • "fix_b" should be 1
  6. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-0_no-graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 0
    • "fix_b" should be 1
  7. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 1
    • "fix_b" should be 1
  8. 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_no-graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 1
    • "fix_b" should be 1
  9. 4-genes_6-edges_artificial-data_Sigmoidal_forward_graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 0
    • "makeGraphs" should be 1
    • "fix_P" should be 0
    • "fix_b" should be 1
  10. 4-genes_6-edges_artificial-data_Sigmoidal_forward_no-graph
    • "Sigmoid" should be 1
    • "estimateParams" should be 0
    • "makeGraphs" should be 0
    • "fix_P" should be 0
    • "fix_b" should be 1
  11. 4-genes_6-edges_artificial-data_MM_estimation_fixP-1_graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 1
    • "fix_b" should be 0
  12. 4-genes_6-edges_artificial-data_MM_estimation_fixP-1_no-graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 1
    • "fix_b" should be 0
  13. 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 1
    • "makeGraphs" should be 1
    • "fix_P" should be 0
    • "fix_b" should be 0
  14. 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_no-graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 1
    • "makeGraphs" should be 0
    • "fix_P" should be 0
    • "fix_b" should be 0
  15. 4-genes_6-edges_artificial-data_MM_forward_graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 0
    • "makeGraphs" should be 1
    • "fix_P" should be 1
    • "fix_b" should be 0
  16. 4-genes_6-edges_artificial-data_MM_forward_no-graph
    • "Sigmoid" should be 0
    • "estimateParams" should be 0
    • "makeGraphs" should be 0
    • "fix_P" should be 1
    • "fix_b" should be 0

Running GRNmap

You will now finally run the GRNmap model on each input workbook created above.

  1. Open MATLAB and navigate to the "matlab" subfolder in the "GRNmap-beta" folder on the desktop
  2. Move the input sheet that you would like to run into this folder.
  3. In MATLAB type the command "GRNmodel" and select the input sheet of study
  4. Once the model has finished running, plots showing the expression over time for all of the genes in the network will appear if "makeGraphs" was set to 1. The plots will automatically be saved as *.jpg files in the same folder as your input file. Compile the figures into a folder following the naming convention described earlier. Compress this folder by right clicking then selecting "7-zip" and "Add to archive...". Make sure the archive format is "zip."
  5. Upload the .xlsx output sheet and the zipped folder with the output plots onto openwetware.
  6. In between each run type "close all" and "clear all" into MATLAB

Updating GRNmap on Github

  1. Download "Git" from this link
  2. Go through the instillation process (no changes needed)
  3. Once installed open "Git Bash" from the programs menu on the computer
  4. To have the GRNmap folder to edit on the desktop type cd Desktop then press enter
  5. To clone into GRNmap type git clone https://github.com/kdahlquist/GRNmap.git then press enter
  6. To edit GRNmap, type cd GRNmap then press enter
  7. To edit the Beta version, type git checkout beta
  8. There will now be a file on the desktop titled "GRNmap" where you can put the updated test files and then upload them to github
  9. Create two new folders in the "test_files" subfolder called "sixteen_tests" and "sixteen_tests_output"
    • The "sixteen_tests" contains the input sheets and the "sixteen_tests_output" contains the output sheet.
  10. To update the GRNmap on the web
    • Type git checkout beta then press enter
    • Type git status then press enter will show the changes that you made in red
    • Type git add test_files/s then press enter
    • Type git add test_files/s then press tab. The code should then read git add test_files/sixteen_tests/ then press space and type git add test_files/s again then tab so it reads git add test_files/sixteen_tests_output/ then type an underscore to indicate you're done.
    • Type git status then press enter
    • Type git commit -m "add sixteen combinations of input files and their output files" or a description of the change that was made then press enter
      • I made a typo and wrote "coombinations instead of combinations", unfortunately
    • In order to configure the computer to your user name Type git config --global user.name "github username" then press enter and then type git config --global user.email "email used for github" then press enter
    • To update anything that was changed while you were working type git pull then press enter
    • To update the website with your changes type git push then press enter and enter your username and password when prompted.
  11. Comment on Issue 74 on Github explaining what was done and change the label to "review requested" and select Dr. Dahlquist.
  12. To remove files git rm test_files/sixteen_tests_output/*.xlsx

Adjust Input Sheets to Correct for Change to Sheet Name

  1. Change sheet names "wt" and "dcin5" to "wt_log2_expression" and "dcin5_log2_expression"
  2. Save and rerun on Matlab
    • When the new workbook with updated sheet names with the newest beta version (4:39 pm)the following Matlab error appeared Error using * Inner matrix dimensions must agree. Error in general_least_squares_error (line 94) L = L + alpha*(wts')*wts + alpha*(b-bp)'*(b-bp) + alpha*sum(proratep(:))^2; Error in lse (line 53) GRNstruct.GRNOutput.lse_0 = general_least_squares_error(initial_guesses); Error in GRNmodel (line 32) GRNstruct = lse(GRNstruct);
    • Alpha was accidentally deleted from the code, but once the error was fixed, try running again.
  3. Upload new input and output sheets into github

Bug found in Optimization Parameters

  1. It was discovered that in the optimization parameters sheet of 4-genes_6-edges_artificial-data_Sigmoid_estimation.xls the times were listed as 0.4, 0.8, 1.2, and 1.4 instead of 04, 0.8, 1.2, and 1.6.
  2. Correct the bug and save as a new document 4-genes_6-edges_artificial-data_Sigmoid_estimation_updated.xlsx (this name may need to be adjusted later).
  3. Run this new file through MATLAB.
  4. There was also some problem with how MATLAB is reading the time points in K1 of the log2_expression sheets. Retyping "1.2" corrected this issue.
  5. Once the input sheets have been corrected run them in MATLAB and upload the input and output sheets to Github. Upload the input, output, .mat, and zipped plots to openwetware.

Check Output Sheet

  1. To check that the output sheets are correct, make an excel checklist with a column with the sixteen input sheet names and then rows with the following labels: wt_log2_optimized_expression, wt_sigmas, dcin5_log2_optimized_expression, dcin5_sigmas, optimized_production_rates, optimized_threshold_b network_optimized_weights, and optimization_diagnostics.
  2. Check each output sheet and write "yes" or "no" when appropriate in the appropriate cell.
  3. Report findings both on the github issue and openwetware.

Identical Runs with Same Code Version and Same Input Workbook (June 2, 2015)

GRNmap Testing Report

GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01

Selecting and Naming the Input Sheets

  1. There were four total input sheets compared for this test. They were taken from the "sixteen_tests" subfolder of the GRNmap-beta branch.
  2. The input sheets taken for study were
    • 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph.xlsx (& copy)
    • 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph.xlsx (& copy)
  3. To simplify the four files to run will be named:
    • 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-1.xlsx
    • 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-2.xlsx
    • 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-1.xlsx
    • 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-2.xlsx

Running the model in MATLAB=

  1. Drag into a folder on the desktop named "GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01" then begin running in MATLAB


When run in MATLAB the following error occurred: Error using inputcheck (line 40) Multiple inputs that look like file names: 'C:\Users\Student\Desktop\GRNmap' and 'Testing'. Error in print (line 156) [pj, devices, options ] = inputcheck( pj, inputargs{:} ); Error in graphs (line 46) eval(['print -djpeg ' directory 'figure_' num2str(kk)]); Error in output (line 6) GRNstruct = graphs(GRNstruct); Error in GRNmodel (line 34) GRNstruct = output(GRNstruct);

  • The problem was that there were spaces in the name of the folder. Rename it to GRNmap_Testing_Report_Identical_Runs_with_Same_Code_Version_and_Same_Input_Workbook_2015-06-01

2. Make sure to save the input file, output file, .mat output sheet, plots, and counter ("figure 1") and upload to openwetware.

Computing LSE and Penalty Term

  1. To get the LSE & the penalty term, type the following into MATLAB
Code for LSE:
GRNstruct.GRNOutput.lse_out

Code for Penalty
GRNstruct.GRNOutput.reg_out

2. Record these values on the wiki, as well as the counter number (which should be saved manually, named "Counter.jpeg" and zipped into the folder with the other plots)

Analysis Workbook

  1. Use an analysis sheet that was constructed for a different test as a basis for how to compare the different copies.
  2. Delete the plots on all three sheets.
  3. Delete the column headers that you do not need for this comparison.
    • The headings should be Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-1; Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-2; MM_estimation_fixP-0_graph_copy-1; MM_estimation_fixP-0_graph_copy-2; Sigmoidal_estimation_fixb-0_fixP-0 _copy-1_vs_copy-2; MM_estimation_fixP-0 _copy-1_vs_copy-2; Sigmoidal_estimation_fixb-0_fixP-0 _copy-1_vs_copy-2_maximum; MM_estimation_fixP-0 _copy-1_vs_copy-2_maximum, starting at column B, leaving column A the same as in the basis analysis sheet.
    • The columns that have copy-1_vs_copy-2 should contain the difference between the copies (=B2-C2 or =D2-E2)
    • The columns that have maximum should take the maximum of the difference (=MAX(F:F) or =MAX(G:G))
  4. In this case the LSE, Penalty term, and number of iterations were exactly the same, but in the future if they are not, this same method can be used to compare them.
  5. Report findings on openwetware and comment results on Issue 99.

Dahlquist Lab Navigation

Retrieved from "https://openwetware.org/mediawiki/index.php?title=Tessa_A._Morris_Electronic_Lab_Notebook&oldid=887450"