Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2645 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.7460. Min Act: -0.1700

Data Index: 1003637 (Open Web Text)

Max Activating Token Index: 18

Text #1

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6582. Min Act: -0.1700

Data Index: 4122178 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6523. Min Act: -0.1700

Data Index: 3935864 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6439. Min Act: -0.1700

Data Index: 556672 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6327. Min Act: -0.1700

Data Index: 6404032 (Open Web Text)

Max Activating Token Index: 19

Text #5

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6142. Min Act: -0.1700

Data Index: 2964708 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.6111. Min Act: -0.1700

Data Index: 3635251 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5910. Min Act: -0.1700

Data Index: 3187283 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5953. Min Act: -0.1700

Data Index: 2530766 (Open Web Text)

Max Activating Token Index: 18

Text #9

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5915. Min Act: -0.1700

Data Index: 4115958 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5898. Min Act: -0.1700

Data Index: 4777200 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5918. Min Act: -0.1700

Data Index: 4784733 (Open Web Text)

Max Activating Token Index: 394

Text #12

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5801. Min Act: -0.1700

Data Index: 6592100 (Open Web Text)

Max Activating Token Index: 17

Text #13

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5683. Min Act: -0.1700

Data Index: 4429652 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5863. Min Act: -0.1700

Data Index: 913664 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5560. Min Act: -0.1700

Data Index: 4291003 (Open Web Text)

Max Activating Token Index: 207

Text #16

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5583. Min Act: -0.1700

Data Index: 3935863 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5445. Min Act: -0.1700

Data Index: 1046072 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5534. Min Act: -0.1700

Data Index: 2218035 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 2.7460. Min Range: -2.7460

Max Act: 2.5518. Min Act: -0.1700

Data Index: 6108803 (Open Web Text)

Max Activating Token Index: 5