Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2798 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4639. Min Act: -0.1700

Data Index: 2672478 (Open Web Text)

Max Activating Token Index: 282

Text #1

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4449. Min Act: -0.1700

Data Index: 4478285 (Open Web Text)

Max Activating Token Index: 169

Text #2

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 5295895 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 2709652 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 4823549 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 6811921 (Open Web Text)

Max Activating Token Index: 2

Text #6

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 6542240 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 4765193 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 682327 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4036. Min Act: -0.1700

Data Index: 8042919 (Open Web Text)

Max Activating Token Index: 963

Text #10

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 7080516 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4099. Min Act: -0.1700

Data Index: 4033980 (Open Web Text)

Max Activating Token Index: 990

Text #12

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 3982566 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.4172. Min Act: -0.1700

Data Index: 4286392 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3888. Min Act: -0.1700

Data Index: 8514826 (Open Web Text)

Max Activating Token Index: 942

Text #15

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3973. Min Act: -0.1700

Data Index: 852353 (Open Web Text)

Max Activating Token Index: 775

Text #16

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3619. Min Act: -0.1700

Data Index: 2165385 (Open Web Text)

Max Activating Token Index: 271

Text #17

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3716. Min Act: -0.1700

Data Index: 2408589 (Open Web Text)

Max Activating Token Index: 42

Text #18

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3771. Min Act: -0.1700

Data Index: 2835901 (Open Web Text)

Max Activating Token Index: 277

Text #19

Max Range: 3.4639. Min Range: -3.4639

Max Act: 3.3526. Min Act: -0.1700

Data Index: 855871 (Open Web Text)

Max Activating Token Index: 315