1. The SCOP profiles database we used is in directory scopdom_v1.69_set4147. If you want to build your own profiles, you can extact the top sequences by yourself.

2. Compare your own profile with the profiles database and store the alignment results into one file. example_alns.aln is the example file of what kind of alignment format we used.

3. Run code cal_gdtts_all.pl and get GDTTS results for all the alignments. For example programs/cal_gdtts_all.pl example_alns.aln example_gdtts.dat >& gdtts.log 

4. Run code get_scores.pl to get the scores lit. For example: programs/get_scores.pl example_alns.aln. The scores list is stored as score_table.dat.

5. Run code mk_sorted_hit_list_recal to sort the result by e_value. For example: mk_sorted_hit_list_recal. The code will read score_table.dat automatically and store the sorted result into file 0sorted_list.dat.

6. Run code cal_accuracy_positive.pl to get accuracy parameter results (accuracy, coverage, ...). For example: programs/cal_accuracy_positive.pl example_alns.aln example_accuracy.dat >& accuracy.log.

7. Exact the Nacc (numbers of correctly aligned positions). For example: cut -f1,2 example_accuracy.dat > example_nacc.dat.

8. Run code mk_score_all_table.pl to calculate all scores and store them into file scores_all_table.dat. For example: programs/mk_score_all_table.pl example_gdtts.dat example_nacc.dat.

9. Run code plot_all_dp_indp.pl to get the True_False table. For example: programs/plot_all_dp_indp.pl. The code will produce 4 tables. plot_all_dependent.dat is the table for Reference Dependent Without Alignment Quality. plot_all_dependent_w_nacc.dat is the table for Reference Dependent With Alignment Quality. plot_global_independent.dat is the table for Reference Independent With Global GDTTS. plot_global_independent.dat is the table for Reference Independent With Local GDTTS. You can use some comercial software to plot the results. We used OriginPro 7 to plot the result ROCs.

10. Run code cal_accuracy_result.pl to get the statistical results for the quality paremeters(accuracy, coverage, Q_modeller, Q_developer and Q_combined). For example: programs/cal_accuracy_result.pl example_accuracy.dat example_accuracy_result.dat. The result is stored as mean and standard error. You can use some comercial software to get bar plot. We used SigmaPlot 10.0.

11. Run codes get_gdtts_results.p and cal_gdtts_result.pl to get gdtts results and do statistical analysis. For example: programs/get_gdtts_results.p and then cal_gdtts_result.pl example_gdtts_p.dat example_gdtts_p_result.dat. Then you can use SigmaPlot to plot the results.

12. Run code get_family_scores.pl to get family results. For example: mkdir scores. programs/get_family_scores.pl. The results are stored into directory scores.

13. Run code roc_family.pl to get family ROC score. For example: programs/roc_family.pl. The result is stored into roc_family.dat.

14. Run code sel_sen_pair.pl to get selectivity_sensitivity scores. For example: programs/sel_sen_pair.pl. The result is stored into sel_sen.dat.

15. If you want to compare the family ROC result between two methods, run code get_roc_pair.pl.

16. If you want to compare the selectivity_sensitivity scores between two methods, run code get_sen_sel_pair.pl. 

17. SVM scores of all the 4147 entries are stored in directory svm_score_4147/. Instead of using scop ID, we used our own ID in this database. The lookout table our ID and scop ID is domain_id.txt.


Thanks for using our evaluation methods, please let us konw if you have any suggestion or advice.