Click the buttons to move through each training waypoint:
In this picture, the white dot circled in red is a small nodule. None of the other dots are nodules.

Bot

  • Nodule detection: 0.00% @Recall1
  • Malignancy detection: 0.00% @Recall1

Radiologist

The AI system will need to learn two different skills: how to detect nodules in the lungs, and how to identify which nodules are likely to be malignant.

Detection:

A nodule is a small lump of tissue that is not normally present in the lungs. Finding nodules is a classic "needle-in-a-haystack" problem. This is because nodules are usually tiny, and many other structures like blood vessels and scars can look like them on first glance.

In this picture, the white dot that is circled in red is a small nodule. None of the other dots are nodules.

Malignancy:

There are a range of nodule features that radiologists use to determine which nodules might be cancerous. The most commonly used system is the Fleischner criteria, which only consider the size and number of nodules, and the presence of risk factors like smoking. Other systems are more complicated, and include the shape and location of the nodules.

A doctor would know for certain the area circled in red is not a nodule, but at this point, the AI still doesn’t get it right.

Bot

  • Nodule detection: 46.40% @Recall1
  • Malignancy detection: 46.32% @Recall1

Radiologist

The AI at this point doesn't really know anything about nodules. Quite often it will highlight blood vessels, saying that it isn't certain they’re nodules but is suspicious they are cancer. Since cancer only occurs in nodules, this makes no sense—it’s an impossible contradiction.

A human, on the other hand, would know this is not a nodule—and therefore, is not cancer, because it’s branching. That’s a clear sign that it’s a blood vessel.

The half-trained AI system thinks this fold in the wall of the large bowel in the abdomen might be a lung nodule.

Bot

  • Nodule detection: 59.80% @Recall1
  • Malignancy detection: 68.71% @Recall1

Radiologist

At this stage, the AI system has very high confidence when detecting large nodules (over 1 cm in diameter). It still hasn't learned some very simple lessons though, and is moderately confident that there are lung nodules in areas of completely normal lung, or even in the bowel. This shows the complete lack of common sense that characterizes modern AI. Even a three-year-old knows the difference between the chest and the belly, but this AI only learns to look for "nodule-like" patterns. It doesn't know what the abdomen is.

This tiny lump on the outer edge of the lung is almost certainly not cancer, as the AI system has started to learn.

Bot

  • Nodule detection: 64.30% @Recall1
  • Malignancy detection: 76.38% @Recall1

Radiologist

The AI is really starting to get the hang of this now. Almost all of the really confident nodule detections are nodules, and are even the lower-confidence detections are "nodule-like." A difficult task, even for humans, is differentiating between nodules and scarring from previous infections. As a general rule, tiny nodules that are at the very edge of the lung are almost always benign and can be ignored. The AI system is getting quite good at this by now. It isn't as convinced that they are actually nodules (which they are not), and it is fairly sure they are not cancer.

The AI thinks the region circled in red is a nodule, but not cancer; it believes the large nodule at the top of the picture, circled in white, is almost certainly a cancer.

Bot

  • Nodule detection: 67.90% @Recall1
  • Malignancy detection: 82.82% @Recall1

Radiologist

Once it has finished training, the model is doing pretty well. If we take only the very high confidence predictions, we find it is identifying most of the large and suspicious nodules.

As we might hope, the model is almost 100% certain that these large nodules are malignant. Radiologists would definitely treat them as cancer until proven otherwise, and the AI system is just as worried.

Once we get into lower-confidence predictions, we see an interesting pattern: The AI system is less certain that the "nodule-like" things in the lungs are actually nodules, but much more importantly, it is almost certain they are not cancer. In many cases the AI is moderately confident there is a nodule, but nearly 100% certain it is benign. While a human would probably not call them "nodules" in the first place, the answer for the patient is the same.

The AI system thinks the region circled in red is a nodule with fairly high confidence, but is almost certainly not cancer. A radiologist would agree; in particular, the fact that it is attached to the edge of the lung, and that it is located behind the aorta is reassuring. On the other hand, the large nodule at the top of the picture (circled in white) is almost certainly a cancer, and the AI system agrees.