Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1897 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.8344. Min Act: -0.1700

Data Index: 5842030 (Open Web Text)

Max Activating Token Index: 902

Text #1

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.7256. Min Act: -0.1700

Data Index: 499840 (Open Web Text)

Max Activating Token Index: 875

Text #2

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.7293. Min Act: -0.1700

Data Index: 3860921 (Open Web Text)

Max Activating Token Index: 616

Text #3

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6838. Min Act: -0.1700

Data Index: 2370175 (Open Web Text)

Max Activating Token Index: 324

Text #4

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6825. Min Act: -0.1700

Data Index: 2710184 (Open Web Text)

Max Activating Token Index: 538

Text #5

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6332. Min Act: -0.1700

Data Index: 7895964 (Open Web Text)

Max Activating Token Index: 683

Text #6

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6489. Min Act: -0.1700

Data Index: 3965161 (Open Web Text)

Max Activating Token Index: 756

Text #7

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6206. Min Act: -0.1700

Data Index: 4650982 (Open Web Text)

Max Activating Token Index: 461

Text #8

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6056. Min Act: -0.1700

Data Index: 6822373 (Open Web Text)

Max Activating Token Index: 439

Text #9

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.5679. Min Act: -0.1700

Data Index: 6220472 (Open Web Text)

Max Activating Token Index: 332

Text #10

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.6011. Min Act: -0.1700

Data Index: 910498 (Open Web Text)

Max Activating Token Index: 887

Text #11

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.5695. Min Act: -0.1700

Data Index: 4974084 (Open Web Text)

Max Activating Token Index: 910

Text #12

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.5566. Min Act: -0.1700

Data Index: 7452349 (Open Web Text)

Max Activating Token Index: 441

Text #13

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.5204. Min Act: -0.1700

Data Index: 3599513 (Open Web Text)

Max Activating Token Index: 713

Text #14

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.5017. Min Act: -0.1700

Data Index: 2215361 (Open Web Text)

Max Activating Token Index: 311

Text #15

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.4705. Min Act: -0.1700

Data Index: 915675 (Open Web Text)

Max Activating Token Index: 351

Text #16

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.4662. Min Act: -0.1700

Data Index: 2580262 (Open Web Text)

Max Activating Token Index: 572

Text #17

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.4789. Min Act: -0.1700

Data Index: 1312124 (Open Web Text)

Max Activating Token Index: 508

Text #18

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.4746. Min Act: -0.1700

Data Index: 2042285 (Open Web Text)

Max Activating Token Index: 307

Text #19

Max Range: 4.8344. Min Range: -4.8344

Max Act: 4.4736. Min Act: -0.1700

Data Index: 1431519 (Open Web Text)

Max Activating Token Index: 219