Tessa A. Morris Electronic Lab Notebook: Difference between revisions
(add LSE and Penalty term) |
(update some of the methods) |
||
Line 203: | Line 203: | ||
#*4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-1.xlsx | #*4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-1.xlsx | ||
#*4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-2.xlsx | #*4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-2.xlsx | ||
==Running the model in MATLAB=== | |||
#Drag into a folder on the desktop named "GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01" then begin running in MATLAB | #Drag into a folder on the desktop named "GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01" then begin running in MATLAB | ||
<br>When run in MATLAB the following error occurred: | |||
When run in MATLAB the following error occurred: | |||
<code>Error using inputcheck (line 40) | <code>Error using inputcheck (line 40) | ||
Multiple inputs that look like file names: | Multiple inputs that look like file names: | ||
'C:\Users\Student\Desktop\GRNmap' and 'Testing'. Error in print (line 156) [pj, devices, options ] = inputcheck( pj, inputargs{:} ); Error in graphs (line 46) eval(['print -djpeg ' directory 'figure_' num2str(kk)]); Error in output (line 6) GRNstruct = graphs(GRNstruct); Error in GRNmodel (line 34) GRNstruct = output(GRNstruct);</code> | 'C:\Users\Student\Desktop\GRNmap' and 'Testing'. Error in print (line 156) [pj, devices, options ] = inputcheck( pj, inputargs{:} ); Error in graphs (line 46) eval(['print -djpeg ' directory 'figure_' num2str(kk)]); Error in output (line 6) GRNstruct = graphs(GRNstruct); Error in GRNmodel (line 34) GRNstruct = output(GRNstruct);</code> | ||
The problem was that there were spaces in the name of the folder. Rename it to GRNmap_Testing_Report_Identical_Runs_with_Same_Code_Version_and_Same_Input_Workbook_2015-06-01 | *The problem was that there were spaces in the name of the folder. Rename it to GRNmap_Testing_Report_Identical_Runs_with_Same_Code_Version_and_Same_Input_Workbook_2015-06-01 | ||
2. Make sure to save the input file, output file, .mat output sheet, plots, and counter ("figure 1") and upload to openwetware. | |||
===Computing LSE and Penalty Term=== | ===Computing LSE and Penalty Term=== | ||
#To get the LSE & the penalty term, type the following into MATLAB | #To get the LSE & the penalty term, type the following into MATLAB | ||
Line 219: | Line 219: | ||
Code for Penalty | Code for Penalty | ||
GRNstruct.GRNOutput.reg_out | GRNstruct.GRNOutput.reg_out | ||
2. Record these values on the wiki, as well as the counter number (which should be saved manually, named "Counter.jpeg" and zipped into the folder with the other plots) | |||
==Analysis Workbook== | |||
#Use a previously constructed analysis sheet as a basis for how to compare the different copies. | |||
[[Media:GRNmap_Testing_Report_16_Test_Files_2015-06-01.xlsx]] | |||
{{Template: Tessa A. Morris ELN}} |
Revision as of 16:28, 2 June 2015
Wednesday (May 27, 2015)
- Continue working on GRNmap Test Inputs
- Discuss future testing directions
Thursday (May 28, 2015)
- Complete GRNmap Test Inputs
- Begin and complete GRNmap Testing Report .xlsx vs. .xls 2015-05-27
- Library Assignment
Monday (June 1, 2015)
- Begin GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01
- Being GRNmap_Testing_Report 16 Test Files 2015-06-01
Tuesday (June 2, 2015)
Add methods
16 Test Files (June 1 - 2, 2015)
GRNmap Testing Page
GRNmap Testing Report 16 Test Files 2015-06-01
Selecting the GRNmap Version and Base Input Sheet
- Download the GRNmap-beta branch version from Github by downloading the zip file.
- Move it to the desktop and unzip by right-clicking on it and choosing 7-zip > Extract here.
- Open the unzipped "GRNmap-beta branch" folder and find the folder "test_files" then navigate to "estimation_tests" and open the file "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls"
- We compared "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls" and "4-genes_6-edges_artificial-data_Sigmoid_forward.xlsx" and found them to contain different data points and different time points, so the "4-genes_6-edges_artificial-data_Sigmoid_estimation.xls" was chosen as the base for the 16 test files.
Editing the Optimization Parameters for each Input
- There are sixteen different input sheets that need to be tested. The "optimization parameters" worksheet is adjusted for each type. For all inputs, the following optimization parameters will be left as they were originally entered
- "alpha" should be 1.00E-10
- "kk_max" should be 1
- "MaxIter" should be 1e08
- "TolFun" should be 1e-10
- "MaxFunEval" should be 1e08
- "TolX" should be 1e-10
- For the parameter "time" (Cell A13), should have "0.4", "0.8", "1.2", and "1.4"
- For the parameter "Strain" (Cell A14), make sure it says "dcin5", making sure that the capitalization and spelling is the same as the worksheet containing that strain's expression data.
- For the parameter "Sheet" (Cell A15), give the number of the worksheet from left to right that your "Strain" log2 expression data is in. This should be the fourth sheet.
- For the parameter "Deletion", leave the zero in cell B15 (corresponding to wt). In cell C15, put a number corresponding to the position in the list of gene names that the gene that was deleted appears. This should be the number three in the list (disregard the column header in this count and only consider the actual gene names themselves).
- For the parameter, "simtime", the row should read: 0, 0.1, 0.2, 0.3 <...fill by steps of 0.1...>, 2, each number in a different cell.
- The parameters "Sigmoid","estimateParams", "makeGraphs","fix_P", and "fix_b" will be different for each input sheet.
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 0
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_no-graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 0
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-1_graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 1
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-1_no-graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 1
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-0_graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 0
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-0_no-graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 0
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 1
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-1_fixP-1_no-graph
- "Sigmoid" should be 1
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 1
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_Sigmoidal_forward_graph
- "Sigmoid" should be 1
- "estimateParams" should be 0
- "makeGraphs" should be 1
- "fix_P" should be 0
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_Sigmoidal_forward_no-graph
- "Sigmoid" should be 1
- "estimateParams" should be 0
- "makeGraphs" should be 0
- "fix_P" should be 0
- "fix_b" should be 1
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-1_graph
- "Sigmoid" should be 0
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 1
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-1_no-graph
- "Sigmoid" should be 0
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 1
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph
- "Sigmoid" should be 0
- "estimateParams" should be 1
- "makeGraphs" should be 1
- "fix_P" should be 0
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_no-graph
- "Sigmoid" should be 0
- "estimateParams" should be 1
- "makeGraphs" should be 0
- "fix_P" should be 0
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_MM_forward_graph
- "Sigmoid" should be 0
- "estimateParams" should be 0
- "makeGraphs" should be 1
- "fix_P" should be 1
- "fix_b" should be 0
- 4-genes_6-edges_artificial-data_MM_forward_no-graph
- "Sigmoid" should be 0
- "estimateParams" should be 0
- "makeGraphs" should be 0
- "fix_P" should be 1
- "fix_b" should be 0
Running GRNmap
You will now finally run the GRNmap model on each input workbook created above.
- Open MATLAB and navigate to the "matlab" subfolder in the "GRNmap-beta" folder on the desktop
- Move the input sheet that you would like to run into this folder.
- In MATLAB type the command "GRNmodel" and select the input sheet of study
- Once the model has finished running, plots showing the expression over time for all of the genes in the network will appear if "makeGraphs" was set to 1. The plots will automatically be saved as
*.jpg
files in the same folder as your input file. Compile the figures into a folder following the naming convention described earlier. Compress this folder by right clicking then selecting "7-zip" and "Add to archive...". Make sure the archive format is "zip." - Upload the .xlsx output sheet and the zipped folder with the output plots onto openwetware.
- In between each run type "close all" and "clear all" into MATLAB
Updating GRNmap on Github
- Download "Git" from this link
- Go through the instillation process (no changes needed)
- Once installed open "Git Bash" from the programs menu on the computer
- To have the GRNmap folder to edit on the desktop type
cd Desktop
then press enter - To clone into GRNmap type
git clone https://github.com/kdahlquist/GRNmap.git
then press enter - To edit GRNmap, type
cd GRNmap
then press enter - To edit the Beta version, type
git checkout beta
- There will now be a file on the desktop titled "GRNmap" where you can put the updated test files and then upload them to github
- Create two new folders in the "test_files" subfolder called "sixteen_tests" and "sixteen_tests_output"
- The "sixteen_tests" contains the input sheets and the "sixteen_tests_output" contains the output sheet.
- To update the GRNmap on the web
- Type
git checkout beta
then press enter - Type
git status
then press enter will show the changes that you made in red - Type
git add test_files/s
then press enter - Type
git add test_files/s
then press tab. The code should then readgit add test_files/sixteen_tests/
then press space and typegit add test_files/s
again then tab so it readsgit add test_files/sixteen_tests_output/
then type an underscore to indicate you're done. - Type
git status
then press enter - Type
git commit -m "add sixteen combinations of input files and their output files"
or a description of the change that was made then press enter- I made a typo and wrote "coombinations instead of combinations", unfortunately
- In order to configure the computer to your user name Type
git config --global user.name "github username"
then press enter and then typegit config --global user.email "email used for github"
then press enter - To update anything that was changed while you were working type
git pull
then press enter - To update the website with your changes type
git push
then press enter and enter your username and password when prompted.
- Type
- Comment on Issue 74 on Github explaining what was done and change the label to "review requested" and select Dr. Dahlquist.
- To remove files
git rm test_files/sixteen_tests_output/*.xlsx
Adjust Input Sheets to Correct for Change to Sheet Name
- Change sheet names "wt" and "dcin5" to "wt_log2_expression" and "dcin5_log2_expression"
- Save and rerun on Matlab
- When the new workbook with updated sheet names with the newest beta version (4:39 pm)the following Matlab error appeared
Error using * Inner matrix dimensions must agree. Error in general_least_squares_error (line 94) L = L + alpha*(wts')*wts + alpha*(b-bp)'*(b-bp) + alpha*sum(proratep(:))^2; Error in lse (line 53) GRNstruct.GRNOutput.lse_0 = general_least_squares_error(initial_guesses); Error in GRNmodel (line 32) GRNstruct = lse(GRNstruct);
- Alpha was accidentally deleted from the code, but once the error was fixed, try running again.
- When the new workbook with updated sheet names with the newest beta version (4:39 pm)the following Matlab error appeared
- Upload new input and output sheets into github
Bug found in Optimization Parameters
- It was discovered that in the optimization parameters sheet of 4-genes_6-edges_artificial-data_Sigmoid_estimation.xls the times were listed as 0.4, 0.8, 1.2, and 1.4 instead of 04, 0.8, 1.2, and 1.6.
- Correct the bug and save as a new document 4-genes_6-edges_artificial-data_Sigmoid_estimation_updated.xlsx (this name may need to be adjusted later).
- Run this new file through MATLAB.
- There was also some problem with how MATLAB is reading the time points in K1 of the log2_expression sheets. Retyping "1.2" corrected this issue.
- Once the input sheets have been corrected run them in MATLAB and upload the input and output sheets to Github. Upload the input, output, .mat, and zipped plots to openwetware.
Check Output Sheet
- To check that the output sheets are correct, make an excel checklist with a column with the sixteen input sheet names and then rows with the following labels: wt_log2_optimized_expression, wt_sigmas, dcin5_log2_optimized_expression, dcin5_sigmas, optimized_production_rates, optimized_threshold_b network_optimized_weights, and optimization_diagnostics.
- Check each output sheet and write "yes" or "no" when appropriate in the appropriate cell.
- Report findings both on the github issue and openwetware.
Identical Runs with Same Code Version and Same Input Workbook (June 2, 2015)
GRNmap Testing Report
GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01
Selecting and Naming the Input Sheets
- There were four total input sheets compared for this test. They were taken from the "sixteen_tests" subfolder of the GRNmap-beta branch.
- The input sheets taken for study were
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph.xlsx (& copy)
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph.xlsx (& copy)
- To simplify the four files to run will be named:
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-1.xlsx
- 4-genes_6-edges_artificial-data_Sigmoidal_estimation_fixb-0_fixP-0_graph_copy-2.xlsx
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-1.xlsx
- 4-genes_6-edges_artificial-data_MM_estimation_fixP-0_graph_copy-2.xlsx
Running the model in MATLAB=
- Drag into a folder on the desktop named "GRNmap Testing Report Identical Runs with Same Code Version and Same Input Workbook 2015-06-01" then begin running in MATLAB
When run in MATLAB the following error occurred:
Error using inputcheck (line 40)
Multiple inputs that look like file names:
'C:\Users\Student\Desktop\GRNmap' and 'Testing'. Error in print (line 156) [pj, devices, options ] = inputcheck( pj, inputargs{:} ); Error in graphs (line 46) eval(['print -djpeg ' directory 'figure_' num2str(kk)]); Error in output (line 6) GRNstruct = graphs(GRNstruct); Error in GRNmodel (line 34) GRNstruct = output(GRNstruct);
- The problem was that there were spaces in the name of the folder. Rename it to GRNmap_Testing_Report_Identical_Runs_with_Same_Code_Version_and_Same_Input_Workbook_2015-06-01
2. Make sure to save the input file, output file, .mat output sheet, plots, and counter ("figure 1") and upload to openwetware.
Computing LSE and Penalty Term
- To get the LSE & the penalty term, type the following into MATLAB
Code for LSE: GRNstruct.GRNOutput.lse_out Code for Penalty GRNstruct.GRNOutput.reg_out
2. Record these values on the wiki, as well as the counter number (which should be saved manually, named "Counter.jpeg" and zipped into the folder with the other plots)
Analysis Workbook
- Use a previously constructed analysis sheet as a basis for how to compare the different copies.
Media:GRNmap_Testing_Report_16_Test_Files_2015-06-01.xlsx
- User Page: Tessa A. Morris
- Personal Lab Notebook: Summer Lab Notebook Lab Notebook 2015-2016
- Lab Home Page: Dahlquist Lab
- Schedule: Summer 2015
- Lionshare