Evidence-guided paper · 2024 · direct-reference · full-text
Protein remote homology detection and structural alignment using deep learning
Tymor Hamamsy; James T. Morton; Robert Blackwell; Daniel Berenberg; Nicholas Carriero; Vladimir Gligorijevic; Charlie E. M. Strauss; Julia Koehler Leman; Kyunghyun Cho; Richard Bonneau. Protein remote homology detection and structural alignment using deep learning. Nature Biotechnology 42:975–985 (2024).
30-second read
TM-Vec predicts TM-score-like similarity from sequence embeddings for remote-homology search, while DeepBLAST uses differentiable DP for alignments.
Central question
Given amino-acid sequences alone, can a system first retrieve structurally similar remote homologs and then predict a structurally meaningful residue alignment?
Intuition
TM-Vec compresses a sequence into a searchable structure-aware vector trained to predict TM-score. DeepBLAST turns residue embeddings into match/gap scores and uses differentiable Needleman–Wunsch to produce an alignment. They solve retrieval and alignment, respectively.
Why it matters
The paper places FAST in a direct manual-reference F1 table: DALI is clearly above FAST; TM-align is numerically higher too, although the MALIDUP gap is only .007 and no significance test is reported; the new DeepBLAST is far below FAST. Thus it shows that DALI beats FAST on this reference-F1 test, not that the new deep model beats FAST, and not that RMSD/coverage is better.
Prerequisites
- Understand residue embeddings and pooled protein embeddings from protein language models.
- Know that TM-score, alignment F1, and RMSD/coverage are different metrics.
- Understand Needleman–Wunsch, match/gap scores, and traceback.
paper-specific guide · plain → technical → input → output → source
Method walkthrough
-
01 · Learn structure-aware vectors from sequences
A protein language model first reads each sequence; a twin network is then trained so cosine similarity between protein vectors approximates the true TM-score.
Technical reading: ProtT5-XL-UniRef50 produces residue embeddings; shared φ transformer layers followed by average pooling, dropout, and fully connected layers output 512-dimensional vectors. Training minimizes L1 distance between cosine similarity and TM-align-derived TM-score.
Input: Protein-sequence pairs and structure-derived TM-scores.
Output: Fixed-length protein embeddings whose similarity predicts TM-score.
Boundary: The supervision comes from TM-align; high embedding similarity is not an actual coordinate superposition.
PDF pp. 12–13, Methods: TM-Vec embedding model
-
02 · Build a vector index and retrieve neighbors
Pre-encode every database sequence as a vector; a new query is encoded once and its nearest protein vectors are retrieved.
Technical reading: Embeddings from SWISS-Prot, CATH, or UniRef50 are indexed with Faiss; cosine similarity drives nearest-neighbor ranking and predicted TM-score, with user-selected top-n output.
Input: A trained TM-Vec model and a sequence database.
Output: Candidate proteins ranked by predicted structural similarity.
Boundary: Retrieval sensitivity and residue-alignment accuracy are separate; finding the right candidate does not align the correct positions.
PDF p. 3, Fig. 1; PDF p. 12, Methods: TM-Vec database creation
-
03 · Predict alignments with differentiable DP
Align top hits residue by residue. Smooth the otherwise nondifferentiable max and traceback operations so alignment errors can backpropagate into learned match and gap rules.
Technical reading: ProtTrans residue embeddings pass through convolutional M/G functions to produce positive match scores μ and negative gap scores g. Needleman–Wunsch max/argmax are smoothed with log-sum-exp/softmax, enabling loss between predicted traceback and ground-truth alignment to train the model; a GPU implementation mitigates the DP bottleneck.
Input: Query-hit sequence pairs and structure alignments for training.
Output: Alignment probabilities/traceback and a sequence-based residue correspondence.
Boundary: DeepBLAST sees sequences only; the paper states that it beats sequence baselines but does not challenge coordinate-based structural aligners.
PDF pp. 12–14, Methods: DeepBLAST alignment module and Differentiable dynamic programming
Key result
On manual-reference F1, DALI scores .791/.619 on MALIDUP/MALISAM versus FAST .569/.300; DeepBLAST itself scores .265/.066 and does not beat FAST.
Evidence-guided deep reading
Paper facts, project readings, and teaching models are labelled separately.
paper-fact
Table 2: who actually beats FAST?
On 234 MALIDUP pairs, F1 is .265 for DeepBLAST, .569 for FAST, .576 for TM-align, and .791 for DALI. On 129 MALISAM pairs, the values are .066, .300, .393, and .619. DALI is clearly highest on both.
The direct answer is that DALI clearly beats FAST on this manual-reference F1 metric. TM-align is numerically higher on both sets, but its MALIDUP result is only .576±.024 versus .569±.026 with no reported significance test, so 'numerically higher' is the justified wording. The new DeepBLAST does not beat FAST. Conflating the paper's 'deep learning' title with the table winner reverses the conclusion.
Source locator: PDF pp. 5–7, Table 2 and Extracting structural alignments from sequence
project-reading
F1 is not RMSD plus coverage
This F1 treats one curated alignment as the reference and scores precision/recall of predicted residue pairs. It does not directly compute post-superposition error in ångströms or aligned-residue coverage.
The authors also note that the 'best alignment' of remote homologs is ambiguous; TM-align sometimes superposes more backbone than the manual reference. Therefore DALI>FAST must be written in full as 'on MALIDUP/MALISAM manual-reference F1.'
Source locator: PDF pp. 6–7, Table 2 legend and Remote homology detection and alignment
paper-fact
Do not treat TM-Vec and DeepBLAST as one output
TM-Vec is mainly validated for TM-score prediction and retrieval: correlation with TM-align TM-score is r=.97 on held-out SWISS-MODEL pairs; median error is .023 on held-out CATH pairs and .042 on held-out folds.
DeepBLAST's main output is a residue alignment. Figure 1 explicitly states that the two were benchmarked separately; a practical pipeline may shortlist with TM-Vec and run DeepBLAST on top-n, but TM-Vec retrieval scores are not DeepBLAST alignment F1.
Source locator: PDF pp. 2–5, Figs. 1–2; PDF p. 7, Remote homology detection and alignment
Study design and evaluation
Data and samples
TM-Vec uses held-out SWISS-MODEL, CATH, and MIP tests; DeepBLAST uses manually curated MALIDUP (234 pairs) and MALISAM (129 pairs), plus a BAGEL application.
Baselines
- Sequence alignment: BLAST, HMMER, Needleman–Wunsch, and Smith–Waterman.
- Structure alignment: Mammoth-local, FAST, TM-align, and DALI.
Metrics
- Alignment F1
- Harmonic mean of precision and recall for predicted residue pairs against a manual reference.
Boundary: Depends on one reference and is not RMSD or coverage. - TM-score prediction error
- Difference between TM-Vec prediction and TM-align-derived TM-score.
Boundary: This approximates a teacher score rather than directly solving coordinates. - Retrieval accuracy/AUPR
- Tests whether nearest neighbors recover the correct CATH class, fold, or homology.
Boundary: Correct retrieval does not guarantee correct residue alignment.
Reported result
DeepBLAST beats all sequence-alignment baselines but trails every coordinate-based method on MALIDUP/MALISAM F1; DALI .791/.619 is clearly above FAST .569/.300, while TM-align .576/.393 is numerically higher without a reported significance test.
PDF pp. 5–7, Table 2 and Fig. 3; PDF pp. 12–15, Methods
teaching-model · not a reported experiment
Teaching example (not a reported experiment)
Correct retrieval can still yield a wrong alignment
Suppose TM-Vec's top hit is truly the same fold, but DeepBLAST recovers 40 of 100 true residue pairs and predicts 20 incorrect pairs.
- Retrieval can be scored top-1 correct; this only says the right protein was found.
- Alignment precision is 40/60, recall 40/100, and F1 about .50; many positions remain wrong.
- Coordinate superposition would produce a third answer via TM-score/RMSD; all three layers belong in separate columns.
Takeaway: Each pipeline stage requires separate validation; retrieval performance cannot validate the aligner.
direct-reference
Evidence boundary versus FAST
It provides a different direct result: DALI beats FAST on manual-reference F1, while the paper's new DeepBLAST does not; this is not an RMSD-plus-coverage claim.
Lawful source and access
24 pages · SHA-256 028cc8ff54aa8862ea800d0a657d433775af56fcc4eb04220750b49f99989cb0
Lawful PMC full text; the 24-page PDF includes the article and reporting checklist.
Limits and misreadings
- Sequence-based retrieval and structural alignment are distinct outputs.
- Manual-reference F1 measures something different from geometric RMSD/coverage.
Source locator map
- PDF pp. 2–4, Figs. 1–2
- PDF pp. 5–7, Table 2 and Fig. 3
- PDF pp. 12–14, Methods: TM-Vec and DeepBLAST
- PDF pp. 14–15, training datasets and model training
Check understanding
Does the paper's new DeepBLAST beat FAST?
Answer: No; DeepBLAST scores .265/.066 on MALIDUP/MALISAM versus FAST's .569/.300.
DALI clearly beats FAST; TM-align is only numerically higher in the table without a reported significance test, while DeepBLAST is lower.
Does DALI's higher F1 than FAST mean lower RMSD and higher coverage?
Answer: No; F1 is agreement with manually referenced residue pairs.
The geometric two-metric claim still requires RMSD and aligned length on the same data.
What do TM-Vec and DeepBLAST each do?
Answer: TM-Vec predicts structural similarity and retrieves candidates; DeepBLAST predicts residue alignments.
Figure 1 links them as two stages while explicitly benchmarking them separately.
Completion task: Redraw Table 2 into sequence-only, coordinate-based, and new-method columns; mark each winner and explain in one sentence why the title cannot substitute for reading the table.
Paper-specific glossary
- twin network
- Two inputs share one encoder and are compared by vector similarity.
- differentiable dynamic programming
- Dynamic programming with smoothed max/argmax so gradients can propagate into model parameters.
- alignment F1
- The harmonic mean of predicted residue-pair precision and recall.
- teacher score
- A score generated by an existing method and used as a learning target; here, TM-align TM-score.