Evidence-guided paper · 2024 · direct-reference · full-text

Protein remote homology detection and structural alignment using deep learning

Tymor Hamamsy; James T. Morton; Robert Blackwell; Daniel Berenberg; Nicholas Carriero; Vladimir Gligorijevic; Charlie E. M. Strauss; Julia Koehler Leman; Kyunghyun Cho; Richard Bonneau. Protein remote homology detection and structural alignment using deep learning. Nature Biotechnology 42:975–985 (2024).

30-second read

TM-Vec predicts TM-score-like similarity from sequence embeddings for remote-homology search, while DeepBLAST uses differentiable DP for alignments.

Central question

Given amino-acid sequences alone, can a system first retrieve structurally similar remote homologs and then predict a structurally meaningful residue alignment?

Intuition

TM-Vec compresses a sequence into a searchable structure-aware vector trained to predict TM-score. DeepBLAST turns residue embeddings into match/gap scores and uses differentiable Needleman–Wunsch to produce an alignment. They solve retrieval and alignment, respectively.

Why it matters

The paper places FAST in a direct manual-reference F1 table: DALI is clearly above FAST; TM-align is numerically higher too, although the MALIDUP gap is only .007 and no significance test is reported; the new DeepBLAST is far below FAST. Thus it shows that DALI beats FAST on this reference-F1 test, not that the new deep model beats FAST, and not that RMSD/coverage is better.

Prerequisites

  • Understand residue embeddings and pooled protein embeddings from protein language models.
  • Know that TM-score, alignment F1, and RMSD/coverage are different metrics.
  • Understand Needleman–Wunsch, match/gap scores, and traceback.

paper-specific guide · plain → technical → input → output → source

Method walkthrough

  1. 01 · Learn structure-aware vectors from sequences

    A protein language model first reads each sequence; a twin network is then trained so cosine similarity between protein vectors approximates the true TM-score.

    Technical reading: ProtT5-XL-UniRef50 produces residue embeddings; shared φ transformer layers followed by average pooling, dropout, and fully connected layers output 512-dimensional vectors. Training minimizes L1 distance between cosine similarity and TM-align-derived TM-score.

    Input: Protein-sequence pairs and structure-derived TM-scores.

    Output: Fixed-length protein embeddings whose similarity predicts TM-score.

    Boundary: The supervision comes from TM-align; high embedding similarity is not an actual coordinate superposition.

    PDF pp. 12–13, Methods: TM-Vec embedding model

  2. 02 · Build a vector index and retrieve neighbors

    Pre-encode every database sequence as a vector; a new query is encoded once and its nearest protein vectors are retrieved.

    Technical reading: Embeddings from SWISS-Prot, CATH, or UniRef50 are indexed with Faiss; cosine similarity drives nearest-neighbor ranking and predicted TM-score, with user-selected top-n output.

    Input: A trained TM-Vec model and a sequence database.

    Output: Candidate proteins ranked by predicted structural similarity.

    Boundary: Retrieval sensitivity and residue-alignment accuracy are separate; finding the right candidate does not align the correct positions.

    PDF p. 3, Fig. 1; PDF p. 12, Methods: TM-Vec database creation

  3. 03 · Predict alignments with differentiable DP

    Align top hits residue by residue. Smooth the otherwise nondifferentiable max and traceback operations so alignment errors can backpropagate into learned match and gap rules.

    Technical reading: ProtTrans residue embeddings pass through convolutional M/G functions to produce positive match scores μ and negative gap scores g. Needleman–Wunsch max/argmax are smoothed with log-sum-exp/softmax, enabling loss between predicted traceback and ground-truth alignment to train the model; a GPU implementation mitigates the DP bottleneck.

    Input: Query-hit sequence pairs and structure alignments for training.

    Output: Alignment probabilities/traceback and a sequence-based residue correspondence.

    Boundary: DeepBLAST sees sequences only; the paper states that it beats sequence baselines but does not challenge coordinate-based structural aligners.

    PDF pp. 12–14, Methods: DeepBLAST alignment module and Differentiable dynamic programming

Key result

On manual-reference F1, DALI scores .791/.619 on MALIDUP/MALISAM versus FAST .569/.300; DeepBLAST itself scores .265/.066 and does not beat FAST.

Evidence-guided deep reading

Paper facts, project readings, and teaching models are labelled separately.

paper-fact

Table 2: who actually beats FAST?

On 234 MALIDUP pairs, F1 is .265 for DeepBLAST, .569 for FAST, .576 for TM-align, and .791 for DALI. On 129 MALISAM pairs, the values are .066, .300, .393, and .619. DALI is clearly highest on both.

The direct answer is that DALI clearly beats FAST on this manual-reference F1 metric. TM-align is numerically higher on both sets, but its MALIDUP result is only .576±.024 versus .569±.026 with no reported significance test, so 'numerically higher' is the justified wording. The new DeepBLAST does not beat FAST. Conflating the paper's 'deep learning' title with the table winner reverses the conclusion.

Source locator: PDF pp. 5–7, Table 2 and Extracting structural alignments from sequence

project-reading

F1 is not RMSD plus coverage

This F1 treats one curated alignment as the reference and scores precision/recall of predicted residue pairs. It does not directly compute post-superposition error in ångströms or aligned-residue coverage.

The authors also note that the 'best alignment' of remote homologs is ambiguous; TM-align sometimes superposes more backbone than the manual reference. Therefore DALI>FAST must be written in full as 'on MALIDUP/MALISAM manual-reference F1.'

Source locator: PDF pp. 6–7, Table 2 legend and Remote homology detection and alignment

paper-fact

Do not treat TM-Vec and DeepBLAST as one output

TM-Vec is mainly validated for TM-score prediction and retrieval: correlation with TM-align TM-score is r=.97 on held-out SWISS-MODEL pairs; median error is .023 on held-out CATH pairs and .042 on held-out folds.

DeepBLAST's main output is a residue alignment. Figure 1 explicitly states that the two were benchmarked separately; a practical pipeline may shortlist with TM-Vec and run DeepBLAST on top-n, but TM-Vec retrieval scores are not DeepBLAST alignment F1.

Source locator: PDF pp. 2–5, Figs. 1–2; PDF p. 7, Remote homology detection and alignment

Study design and evaluation

Data and samples

TM-Vec uses held-out SWISS-MODEL, CATH, and MIP tests; DeepBLAST uses manually curated MALIDUP (234 pairs) and MALISAM (129 pairs), plus a BAGEL application.

Baselines

  • Sequence alignment: BLAST, HMMER, Needleman–Wunsch, and Smith–Waterman.
  • Structure alignment: Mammoth-local, FAST, TM-align, and DALI.

Metrics

Alignment F1
Harmonic mean of precision and recall for predicted residue pairs against a manual reference.
Boundary: Depends on one reference and is not RMSD or coverage.
TM-score prediction error
Difference between TM-Vec prediction and TM-align-derived TM-score.
Boundary: This approximates a teacher score rather than directly solving coordinates.
Retrieval accuracy/AUPR
Tests whether nearest neighbors recover the correct CATH class, fold, or homology.
Boundary: Correct retrieval does not guarantee correct residue alignment.

Reported result

DeepBLAST beats all sequence-alignment baselines but trails every coordinate-based method on MALIDUP/MALISAM F1; DALI .791/.619 is clearly above FAST .569/.300, while TM-align .576/.393 is numerically higher without a reported significance test.

PDF pp. 5–7, Table 2 and Fig. 3; PDF pp. 12–15, Methods

teaching-model · not a reported experiment

Teaching example (not a reported experiment)

Correct retrieval can still yield a wrong alignment

Suppose TM-Vec's top hit is truly the same fold, but DeepBLAST recovers 40 of 100 true residue pairs and predicts 20 incorrect pairs.

  1. Retrieval can be scored top-1 correct; this only says the right protein was found.
  2. Alignment precision is 40/60, recall 40/100, and F1 about .50; many positions remain wrong.
  3. Coordinate superposition would produce a third answer via TM-score/RMSD; all three layers belong in separate columns.

Takeaway: Each pipeline stage requires separate validation; retrieval performance cannot validate the aligner.

direct-reference

Evidence boundary versus FAST

It provides a different direct result: DALI beats FAST on manual-reference F1, while the paper's new DeepBLAST does not; this is not an RMSD-plus-coverage claim.

Lawful source and access

24 pages · SHA-256 028cc8ff54aa8862ea800d0a657d433775af56fcc4eb04220750b49f99989cb0

Lawful PMC full text; the 24-page PDF includes the article and reporting checklist.

Europe PMC author manuscript

Limits and misreadings

  • Sequence-based retrieval and structural alignment are distinct outputs.
  • Manual-reference F1 measures something different from geometric RMSD/coverage.

Source locator map

  1. PDF pp. 2–4, Figs. 1–2
  2. PDF pp. 5–7, Table 2 and Fig. 3
  3. PDF pp. 12–14, Methods: TM-Vec and DeepBLAST
  4. PDF pp. 14–15, training datasets and model training

Check understanding

  1. Does the paper's new DeepBLAST beat FAST?

    Answer: No; DeepBLAST scores .265/.066 on MALIDUP/MALISAM versus FAST's .569/.300.

    DALI clearly beats FAST; TM-align is only numerically higher in the table without a reported significance test, while DeepBLAST is lower.

  2. Does DALI's higher F1 than FAST mean lower RMSD and higher coverage?

    Answer: No; F1 is agreement with manually referenced residue pairs.

    The geometric two-metric claim still requires RMSD and aligned length on the same data.

  3. What do TM-Vec and DeepBLAST each do?

    Answer: TM-Vec predicts structural similarity and retrieves candidates; DeepBLAST predicts residue alignments.

    Figure 1 links them as two stages while explicitly benchmarking them separately.

Completion task: Redraw Table 2 into sequence-only, coordinate-based, and new-method columns; mark each winner and explain in one sentence why the title cannot substitute for reading the table.

Paper-specific glossary

twin network
Two inputs share one encoder and are compared by vector similarity.
differentiable dynamic programming
Dynamic programming with smoothed max/argmax so gradients can propagate into model parameters.
alignment F1
The harmonic mean of predicted residue-pair precision and recall.
teacher score
A score generated by an existing method and used as a learning target; here, TM-align TM-score.