Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1200 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.5666. Min Act: -0.1700

Data Index: 3265505 (Open Web Text)

Max Activating Token Index: 337

Text #1

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.5262. Min Act: -0.1700

Data Index: 1276267 (Open Web Text)

Max Activating Token Index: 439

Text #2

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.5959. Min Act: -0.1700

Data Index: 2890596 (Open Web Text)

Max Activating Token Index: 749

Text #3

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.5431. Min Act: -0.1700

Data Index: 3725119 (Open Web Text)

Max Activating Token Index: 711

Text #4

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.4863. Min Act: -0.1700

Data Index: 4049237 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.4453. Min Act: -0.1700

Data Index: 3259569 (Open Web Text)

Max Activating Token Index: 369

Text #6

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.4345. Min Act: -0.1700

Data Index: 4167311 (Open Web Text)

Max Activating Token Index: 989

Text #7

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.4262. Min Act: -0.1700

Data Index: 2991574 (Open Web Text)

Max Activating Token Index: 371

Text #8

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.4059. Min Act: -0.1700

Data Index: 7595436 (Open Web Text)

Max Activating Token Index: 99

Text #9

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3825. Min Act: -0.1700

Data Index: 4213442 (Open Web Text)

Max Activating Token Index: 128

Text #10

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3660. Min Act: -0.1700

Data Index: 2965793 (Open Web Text)

Max Activating Token Index: 779

Text #11

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3839. Min Act: -0.1700

Data Index: 691999 (Open Web Text)

Max Activating Token Index: 141

Text #12

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3367. Min Act: -0.1700

Data Index: 6787827 (Open Web Text)

Max Activating Token Index: 562

Text #13

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3452. Min Act: -0.1700

Data Index: 1351036 (Open Web Text)

Max Activating Token Index: 994

Text #14

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3237. Min Act: -0.1700

Data Index: 5375062 (Open Web Text)

Max Activating Token Index: 635

Text #15

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3348. Min Act: -0.1700

Data Index: 7245067 (Open Web Text)

Max Activating Token Index: 477

Text #16

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3194. Min Act: -0.1700

Data Index: 2950574 (Open Web Text)

Max Activating Token Index: 722

Text #17

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3162. Min Act: -0.1700

Data Index: 5152803 (Open Web Text)

Max Activating Token Index: 953

Text #18

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3739. Min Act: -0.1700

Data Index: 6867298 (Open Web Text)

Max Activating Token Index: 892

Text #19

Max Range: 4.5959. Min Range: -4.5959

Max Act: 4.3526. Min Act: -0.1700

Data Index: 6789273 (Open Web Text)

Max Activating Token Index: 222