Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1074 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.2263. Min Act: -0.1700

Data Index: 5687974 (Open Web Text)

Max Activating Token Index: 124

Text #1

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1996. Min Act: -0.1700

Data Index: 4584422 (Open Web Text)

Max Activating Token Index: 768

Text #2

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1753. Min Act: -0.1700

Data Index: 6039935 (Open Web Text)

Max Activating Token Index: 343

Text #3

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1675. Min Act: -0.1700

Data Index: 156430 (Open Web Text)

Max Activating Token Index: 578

Text #4

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1955. Min Act: -0.1700

Data Index: 4793290 (Open Web Text)

Max Activating Token Index: 54

Text #5

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1721. Min Act: -0.1700

Data Index: 3955992 (Open Web Text)

Max Activating Token Index: 630

Text #6

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1534. Min Act: -0.1700

Data Index: 1798476 (Open Web Text)

Max Activating Token Index: 261

Text #7

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1301. Min Act: -0.1700

Data Index: 4268107 (Open Web Text)

Max Activating Token Index: 615

Text #8

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1234. Min Act: -0.1700

Data Index: 7097608 (Open Web Text)

Max Activating Token Index: 233

Text #9

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1496. Min Act: -0.1700

Data Index: 4668900 (Open Web Text)

Max Activating Token Index: 576

Text #10

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1441. Min Act: -0.1700

Data Index: 2111941 (Open Web Text)

Max Activating Token Index: 991

Text #11

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1438. Min Act: -0.1700

Data Index: 2073938 (Open Web Text)

Max Activating Token Index: 412

Text #12

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1500. Min Act: -0.1700

Data Index: 1485140 (Open Web Text)

Max Activating Token Index: 365

Text #13

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1030. Min Act: -0.1700

Data Index: 5835039 (Open Web Text)

Max Activating Token Index: 536

Text #14

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1120. Min Act: -0.1700

Data Index: 8180758 (Open Web Text)

Max Activating Token Index: 111

Text #15

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1129. Min Act: -0.1700

Data Index: 7692360 (Open Web Text)

Max Activating Token Index: 460

Text #16

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.1333. Min Act: -0.1700

Data Index: 646221 (Open Web Text)

Max Activating Token Index: 958

Text #17

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.0883. Min Act: -0.1700

Data Index: 2516738 (Open Web Text)

Max Activating Token Index: 866

Text #18

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.0815. Min Act: -0.1700

Data Index: 3991436 (Open Web Text)

Max Activating Token Index: 164

Text #19

Max Range: 5.2263. Min Range: -5.2263

Max Act: 5.0989. Min Act: -0.1700

Data Index: 2897923 (Open Web Text)

Max Activating Token Index: 862