Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1658 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.5643. Min Act: -0.1700

Data Index: 943702 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.5951. Min Act: -0.1700

Data Index: 6578792 (Open Web Text)

Max Activating Token Index: 715

Text #2

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4803. Min Act: -0.1700

Data Index: 6189209 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4779. Min Act: -0.1700

Data Index: 2570993 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4694. Min Act: -0.1700

Data Index: 8721715 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4559. Min Act: -0.1700

Data Index: 3461080 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4405. Min Act: -0.1700

Data Index: 4851930 (Open Web Text)

Max Activating Token Index: 10

Text #7

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4472. Min Act: -0.1700

Data Index: 6639236 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4535. Min Act: -0.1700

Data Index: 6129522 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4384. Min Act: -0.1700

Data Index: 2333422 (Open Web Text)

Max Activating Token Index: 37

Text #10

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4239. Min Act: -0.1700

Data Index: 6801419 (Open Web Text)

Max Activating Token Index: 806

Text #11

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4293. Min Act: -0.1700

Data Index: 1451410 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4080. Min Act: -0.1700

Data Index: 4092473 (Open Web Text)

Max Activating Token Index: 246

Text #13

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4353. Min Act: -0.1700

Data Index: 5279632 (Open Web Text)

Max Activating Token Index: 750

Text #14

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4362. Min Act: -0.1700

Data Index: 5295331 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.3842. Min Act: -0.1700

Data Index: 5150794 (Open Web Text)

Max Activating Token Index: 234

Text #16

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.3813. Min Act: -0.1700

Data Index: 5330977 (Open Web Text)

Max Activating Token Index: 842

Text #17

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4120. Min Act: -0.1700

Data Index: 1138111 (Open Web Text)

Max Activating Token Index: 704

Text #18

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.4113. Min Act: -0.1700

Data Index: 5390451 (Open Web Text)

Max Activating Token Index: 23

Text #19

Max Range: 3.5951. Min Range: -3.5951

Max Act: 3.3951. Min Act: -0.1700

Data Index: 5549877 (Open Web Text)

Max Activating Token Index: 315