何が問題なのか一つわかった。
WARNING: Ignoring invalid symbol '*' at pos. 492 in line 2 of /tmp/tmp2hwa2yuo/query.a3m
これだ。
なんとなく自分の持っているアミノ酸データでFASTAファイルを自分で作って投げていたわけだが
ストップコドンのところのアミノ酸を習慣的にアスタリスクにしてあった。
それがお気に召さなかったようで、アスタリスクを消したFASTAファイルを作ったら前回エラーが出たポイントをクリアした。
ERRORじゃなくてWARNINGだったのでスルーしてたわ。
ということで気を取り直して
ついでに一旦Docker上の環境をやめて直にインストールでやってみる。GPUも手に入ったのでそれも使って
/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/absl/flags/_validators.py:203: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line! warnings.warn( I0826 20:34:58.412197 140720169187136 templates.py:836] Using precomputed obsolete pdbs /mnt/HDD2/af_database//pdb_mmcif/obsolete.dat. I0826 20:34:59.711968 140720169187136 xla_bridge.py:236] Unable to initialize backend 'tpu_driver': Not found: Unable to find driver in registry given worker: I0826 20:34:59.772896 140720169187136 xla_bridge.py:236] Unable to initialize backend 'tpu': Invalid argument: TpuPlatform is not available. I0826 20:35:00.382656 140720169187136 run_alphafold.py:267] Have 1 models: ['model_1'] I0826 20:35:00.382773 140720169187136 run_alphafold.py:280] Using random seed 8088228927505236060 for the data pipeline I0826 20:35:00.383199 140720169187136 jackhmmer.py:130] Launching subprocess "/home/kuro/miniconda3/envs/alphafold/bin/jackhmmer -o /dev/null -A /tmp/tmpwx4x1v4e/output.sto --noali --F1 0.0005 --F2 5e-05 --F3 5e-07 --incE 0.0001 -E 0.0001 --cpu 8 -N 1 /home/kuro/test5.fasta /mnt/HDD2/af_database//uniref90/uniref90.fasta" I0826 20:35:00.406429 140720169187136 utils.py:36] Started Jackhmmer (uniref90.fasta) query I0826 20:40:26.839800 140720169187136 utils.py:40] Finished Jackhmmer (uniref90.fasta) query in 326.433 seconds I0826 20:40:26.901696 140720169187136 jackhmmer.py:130] Launching subprocess "/home/kuro/miniconda3/envs/alphafold/bin/jackhmmer -o /dev/null -A /tmp/tmp86y_pxlt/output.sto --noali --F1 0.0005 --F2 5e-05 --F3 5e-07 --incE 0.0001 -E 0.0001 --cpu 8 -N 1 /home/kuro/test5.fasta /mnt/HDD2/af_database//mgnify/mgy_clusters.fa" I0826 20:40:26.917324 140720169187136 utils.py:36] Started Jackhmmer (mgy_clusters.fa) query I0826 20:45:57.057168 140720169187136 utils.py:40] Finished Jackhmmer (mgy_clusters.fa) query in 330.140 seconds I0826 20:45:58.580264 140720169187136 hhsearch.py:76] Launching subprocess "/home/kuro/miniconda3/envs/alphafold/bin/hhsearch -i /tmp/tmpkl7ayr4a/query.a3m -o /tmp/tmpkl7ayr4a/output.hhr -maxseq 1000000 -d /mnt/HDD2/af_database//pdb70/pdb70" I0826 20:45:58.635203 140720169187136 utils.py:36] Started HHsearch query I0826 20:50:30.194164 140720169187136 utils.py:40] Finished HHsearch query in 271.558 seconds I0826 20:50:33.453487 140720169187136 hhblits.py:128] Launching subprocess "/home/kuro/miniconda3/envs/alphafold/bin/hhblits -i /home/kuro/test5.fasta -cpu 4 -oa3m /tmp/tmpkobbonvm/output.a3m -o /dev/null -n 3 -e 0.001 -maxseq 1000000 -realign_max 100000 -maxfilt 100000 -min_prefilter_hits 1000 -d /mnt/HDD2/af_database//bfd/bfd_metaclust_clu_complete_id30_c90_final_seq.sorted_opt -d /mnt/HDD2/af_database//uniclust30/uniclust30_2018_08/uniclust30_2018_08" I0826 20:50:33.492181 140720169187136 utils.py:36] Started HHblits query I0827 02:29:46.357933 140720169187136 utils.py:40] Finished HHblits query in 20352.865 seconds
このへんまでは順調。
て、ここまでで６時間位かかってるけど。このへんはCPU勝負らしいのでもうちょっとCPUスペックをあげるべきなのかも。
んでもってこのあとテンプレートを探す作業が始まり
I0827 02:29:46.503336 140720169187136 templates.py:848] Searching for template for: None W0827 02:29:46.504062 140720169187136 templates.py:131] Template structure not in release dates dict: 1g87 I0827 02:29:46.504361 140720169187136 templates.py:715] Reading PDB entry from /mnt/HDD2/af_database//pdb_mmcif/mmcif_files/1g87.cif. Query: MAFR・・・・
I0827 02:29:56.245360 140720169187136 templates.py:270] Found an exact template match 3rx5_A.
I0827 02:29:57.180096 140720169187136 pipeline.py:200] Uniref90 MSA size: 8133 sequences.
I0827 02:29:57.180236 140720169187136 pipeline.py:201] BFD MSA size: 2683 sequences.
I0827 02:29:57.180323 140720169187136 pipeline.py:202] MGnify MSA size: 501 sequences.
I0827 02:29:57.180396 140720169187136 pipeline.py:203] Final (deduplicated) MSA size: 11174 sequences.
I0827 02:29:57.180535 140720169187136 pipeline.py:205] Total number of templates (NB: this can include bad templates and is later filtered to top 4): 20.
I0827 02:29:58.202671 140720169187136 run_alphafold.py:142] Running model model_1
I0827 02:30:05.643637 140720169187136 model.py:131] Running predict with shape(feat) = {'aatype': (4, 492), 'residue_index': (4, 492), 'seq_length': (4,), 'template_aatype': (4, 4, 492), 'template_all_atom_masks': (4, 4, 492, 37), 'template_all_atom_positions': (4, 4, 492, 37, 3), 'template_sum_probs': (4, 4, 1), 'is_distillation': (4,), 'seq_mask': (4, 492), 'msa_mask': (4, 508, 492), 'msa_row_mask': (4, 508), 'random_crop_to_size_seed': (4, 2), 'template_mask': (4, 4), 'template_pseudo_beta': (4, 4, 492, 3), 'template_pseudo_beta_mask': (4, 4, 492), 'atom14_atom_exists': (4, 492, 14), 'residx_atom14_to_atom37': (4, 492, 14), 'residx_atom37_to_atom14': (4, 492, 37), 'atom37_atom_exists': (4, 492, 37), 'extra_msa': (4, 5120, 492), 'extra_msa_mask': (4, 5120, 492), 'extra_msa_row_mask': (4, 5120), 'bert_mask': (4, 508, 492), 'true_msa': (4, 508, 492), 'extra_has_deletion': (4, 5120, 492), 'extra_deletion_value': (4, 5120, 492), 'msa_feat': (4, 508, 492, 49), 'target_feat': (4, 492, 22)}
2021-08-27 02:30:06.870769: W external/org_tensorflow/tensorflow/stream_executor/gpu/asm_compiler.cc:81] Couldn't get ptxas version string: Internal: Running ptxas --version returned 32512
2021-08-27 02:30:06.893457: F external/org_tensorflow/tensorflow/compiler/xla/service/gpu/nvptx_compiler.cc:479] ptxas returned an error during compilation of ptx to sass: 'Internal: ptxas exited with non-zero error code 32512, output: ' If the error message indicates that a file could not be written, please verify that sufficient filesystem space is provided.
Fatal Python error: Aborted
Thread 0x00007ffbf7b27740 (most recent call first):
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/interpreters/xla.py", line 385 in backend_compile
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/interpreters/xla.py", line 322 in xla_primitive_callable
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/_src/util.py", line 179 in cached
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/_src/util.py", line 186 in wrapper
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/interpreters/xla.py", line 273 in apply_primitive
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/core.py", line 610 in process_primitive
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/core.py", line 265 in bind
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/_src/lax/lax.py", line 386 in shift_right_logical
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/jax/_src/random.py", line 75 in PRNGKey
File "/home/kuro/alphafold/alphafold/model/model.py", line 133 in predict
File "/home/kuro/alphafold/run_alphafold.py", line 149 in predict_structure
File "/home/kuro/alphafold/run_alphafold.py", line 284 in main
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/absl/app.py", line 258 in _run_main
File "/home/kuro/miniconda3/envs/alphafold/lib/python3.8/site-packages/absl/app.py", line 312 in run
File "/home/kuro/alphafold/run_alphafold.py", line 310 in <module>
うわあ〜最後の最後でまたエラーが出て止まってやがる。なんかTensorflowのバージョン不整合っぽいな。
もうちょっとか。
しかし、６時間かけてエラーは辛すぎ。
もうちょっと小さい分子でテストすべきだな。
