Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 239 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.4674. Min Act: -0.1700

Data Index: 914371 (Open Web Text)

Max Activating Token Index: 737

Text #1

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.4368. Min Act: -0.1700

Data Index: 7300875 (Open Web Text)

Max Activating Token Index: 48

Text #2

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.3365. Min Act: -0.1700

Data Index: 4598878 (Open Web Text)

Max Activating Token Index: 55

Text #3

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.3731. Min Act: -0.1700

Data Index: 156151 (Open Web Text)

Max Activating Token Index: 243

Text #4

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.3498. Min Act: -0.1700

Data Index: 7564124 (Open Web Text)

Max Activating Token Index: 326

Text #5

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.3177. Min Act: -0.1700

Data Index: 6751983 (Open Web Text)

Max Activating Token Index: 33

Text #6

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2912. Min Act: -0.1700

Data Index: 7931654 (Open Web Text)

Max Activating Token Index: 274

Text #7

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.3074. Min Act: -0.1700

Data Index: 3363276 (Open Web Text)

Max Activating Token Index: 36

Text #8

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2186. Min Act: -0.1700

Data Index: 1025681 (Open Web Text)

Max Activating Token Index: 887

Text #9

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2536. Min Act: -0.1700

Data Index: 5412789 (Open Web Text)

Max Activating Token Index: 371

Text #10

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2131. Min Act: -0.1700

Data Index: 2032195 (Open Web Text)

Max Activating Token Index: 265

Text #11

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2220. Min Act: -0.1700

Data Index: 7931656 (Open Web Text)

Max Activating Token Index: 193

Text #12

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2299. Min Act: -0.1700

Data Index: 4729194 (Open Web Text)

Max Activating Token Index: 523

Text #13

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2183. Min Act: -0.1700

Data Index: 4600343 (Open Web Text)

Max Activating Token Index: 58

Text #14

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2106. Min Act: -0.1700

Data Index: 4598876 (Open Web Text)

Max Activating Token Index: 918

Text #15

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.2087. Min Act: -0.1700

Data Index: 3910378 (Open Web Text)

Max Activating Token Index: 456

Text #16

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.1882. Min Act: -0.1700

Data Index: 3058022 (Open Web Text)

Max Activating Token Index: 66

Text #17

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.1965. Min Act: -0.1700

Data Index: 6163947 (Open Web Text)

Max Activating Token Index: 56

Text #18

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.1504. Min Act: -0.1700

Data Index: 808632 (Open Web Text)

Max Activating Token Index: 164

Text #19

Max Range: 5.4674. Min Range: -5.4674

Max Act: 5.1531. Min Act: -0.1700

Data Index: 4315741 (Open Web Text)

Max Activating Token Index: 419