Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2978 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5958. Min Act: -0.1700

Data Index: 6740847 (Open Web Text)

Max Activating Token Index: 138

Text #1

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5822. Min Act: -0.1700

Data Index: 4115312 (Open Web Text)

Max Activating Token Index: 246

Text #2

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5538. Min Act: -0.1700

Data Index: 6592433 (Open Web Text)

Max Activating Token Index: 176

Text #3

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5559. Min Act: -0.1700

Data Index: 502792 (Open Web Text)

Max Activating Token Index: 211

Text #4

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5324. Min Act: -0.1700

Data Index: 5940308 (Open Web Text)

Max Activating Token Index: 213

Text #5

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5092. Min Act: -0.1700

Data Index: 2139872 (Open Web Text)

Max Activating Token Index: 229

Text #6

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.5322. Min Act: -0.1700

Data Index: 1920138 (Open Web Text)

Max Activating Token Index: 457

Text #7

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4898. Min Act: -0.1700

Data Index: 4783026 (Open Web Text)

Max Activating Token Index: 589

Text #8

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4842. Min Act: -0.1700

Data Index: 3100649 (Open Web Text)

Max Activating Token Index: 498

Text #9

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4910. Min Act: -0.1700

Data Index: 7165939 (Open Web Text)

Max Activating Token Index: 380

Text #10

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4701. Min Act: -0.1700

Data Index: 4745008 (Open Web Text)

Max Activating Token Index: 385

Text #11

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4956. Min Act: -0.1700

Data Index: 320612 (Open Web Text)

Max Activating Token Index: 251

Text #12

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4753. Min Act: -0.1700

Data Index: 2729240 (Open Web Text)

Max Activating Token Index: 291

Text #13

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4531. Min Act: -0.1700

Data Index: 5934869 (Open Web Text)

Max Activating Token Index: 371

Text #14

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4537. Min Act: -0.1700

Data Index: 342451 (Open Web Text)

Max Activating Token Index: 241

Text #15

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4632. Min Act: -0.1700

Data Index: 2442999 (Open Web Text)

Max Activating Token Index: 283

Text #16

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4671. Min Act: -0.1700

Data Index: 5427267 (Open Web Text)

Max Activating Token Index: 294

Text #17

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4574. Min Act: -0.1700

Data Index: 4581813 (Open Web Text)

Max Activating Token Index: 555

Text #18

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4679. Min Act: -0.1700

Data Index: 247013 (Open Web Text)

Max Activating Token Index: 333

Text #19

Max Range: 4.5958. Min Range: -4.5958

Max Act: 4.4525. Min Act: -0.1700

Data Index: 4186153 (Open Web Text)

Max Activating Token Index: 68