Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2105 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.3123. Min Act: -0.1700

Data Index: 4466296 (Open Web Text)

Max Activating Token Index: 177

Text #1

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.3220. Min Act: -0.1700

Data Index: 7471374 (Open Web Text)

Max Activating Token Index: 560

Text #2

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.1576. Min Act: -0.1700

Data Index: 1121416 (Open Web Text)

Max Activating Token Index: 664

Text #3

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.1018. Min Act: -0.1700

Data Index: 2204168 (Open Web Text)

Max Activating Token Index: 29

Text #4

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.0929. Min Act: -0.1700

Data Index: 2966416 (Open Web Text)

Max Activating Token Index: 216

Text #5

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.0423. Min Act: -0.1700

Data Index: 1884757 (Open Web Text)

Max Activating Token Index: 715

Text #6

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.0684. Min Act: -0.1700

Data Index: 7720816 (Open Web Text)

Max Activating Token Index: 63

Text #7

Max Range: 3.3220. Min Range: -3.3220

Max Act: 3.0187. Min Act: -0.1700

Data Index: 593067 (Open Web Text)

Max Activating Token Index: 514

Text #8

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9762. Min Act: -0.1700

Data Index: 8735705 (Open Web Text)

Max Activating Token Index: 50

Text #9

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9860. Min Act: -0.1700

Data Index: 4340275 (Open Web Text)

Max Activating Token Index: 106

Text #10

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9720. Min Act: -0.1700

Data Index: 867960 (Open Web Text)

Max Activating Token Index: 852

Text #11

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9636. Min Act: -0.1700

Data Index: 1692526 (Open Web Text)

Max Activating Token Index: 218

Text #12

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9484. Min Act: -0.1700

Data Index: 427711 (Open Web Text)

Max Activating Token Index: 457

Text #13

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9731. Min Act: -0.1700

Data Index: 8174055 (Open Web Text)

Max Activating Token Index: 558

Text #14

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9391. Min Act: -0.1700

Data Index: 8653377 (Open Web Text)

Max Activating Token Index: 32

Text #15

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9232. Min Act: -0.1700

Data Index: 2892701 (Open Web Text)

Max Activating Token Index: 370

Text #16

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9405. Min Act: -0.1700

Data Index: 6904329 (Open Web Text)

Max Activating Token Index: 18

Text #17

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9326. Min Act: -0.1700

Data Index: 8446334 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9494. Min Act: -0.1700

Data Index: 1152731 (Open Web Text)

Max Activating Token Index: 895

Text #19

Max Range: 3.3220. Min Range: -3.3220

Max Act: 2.9432. Min Act: -0.1700

Data Index: 6135944 (Open Web Text)

Max Activating Token Index: 835