Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2874 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1720. Min Range: -4.1720

Max Act: 4.1720. Min Act: -0.1700

Data Index: 1132267 (Open Web Text)

Max Activating Token Index: 331

Text #1

Max Range: 4.1720. Min Range: -4.1720

Max Act: 4.0721. Min Act: -0.1700

Data Index: 7822372 (Open Web Text)

Max Activating Token Index: 1013

Text #2

Max Range: 4.1720. Min Range: -4.1720

Max Act: 4.0621. Min Act: -0.1700

Data Index: 6233708 (Open Web Text)

Max Activating Token Index: 710

Text #3

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9519. Min Act: -0.1700

Data Index: 5971819 (Open Web Text)

Max Activating Token Index: 62

Text #4

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9381. Min Act: -0.1700

Data Index: 4179949 (Open Web Text)

Max Activating Token Index: 271

Text #5

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9378. Min Act: -0.1700

Data Index: 8338177 (Open Web Text)

Max Activating Token Index: 1003

Text #6

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9326. Min Act: -0.1700

Data Index: 3232123 (Open Web Text)

Max Activating Token Index: 585

Text #7

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9344. Min Act: -0.1700

Data Index: 1187069 (Open Web Text)

Max Activating Token Index: 167

Text #8

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9475. Min Act: -0.1700

Data Index: 6922287 (Open Web Text)

Max Activating Token Index: 762

Text #9

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9288. Min Act: -0.1700

Data Index: 4054406 (Open Web Text)

Max Activating Token Index: 333

Text #10

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.9010. Min Act: -0.1700

Data Index: 4963173 (Open Web Text)

Max Activating Token Index: 782

Text #11

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8597. Min Act: -0.1700

Data Index: 3624135 (Open Web Text)

Max Activating Token Index: 85

Text #12

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8743. Min Act: -0.1700

Data Index: 5762145 (Open Web Text)

Max Activating Token Index: 895

Text #13

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8683. Min Act: -0.1700

Data Index: 599393 (Open Web Text)

Max Activating Token Index: 889

Text #14

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8673. Min Act: -0.1700

Data Index: 4576792 (Open Web Text)

Max Activating Token Index: 331

Text #15

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8259. Min Act: -0.1700

Data Index: 4889049 (Open Web Text)

Max Activating Token Index: 185

Text #16

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8287. Min Act: -0.1700

Data Index: 847080 (Open Web Text)

Max Activating Token Index: 647

Text #17

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8279. Min Act: -0.1700

Data Index: 1471171 (Open Web Text)

Max Activating Token Index: 269

Text #18

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8343. Min Act: -0.1700

Data Index: 2944087 (Open Web Text)

Max Activating Token Index: 198

Text #19

Max Range: 4.1720. Min Range: -4.1720

Max Act: 3.8539. Min Act: -0.1700

Data Index: 4200543 (Open Web Text)

Max Activating Token Index: 154