Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 833 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5237. Min Act: -0.1700

Data Index: 4634148 (Open Web Text)

Max Activating Token Index: 358

Text #1

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5063. Min Act: -0.1700

Data Index: 3627071 (Open Web Text)

Max Activating Token Index: 987

Text #2

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5087. Min Act: -0.1700

Data Index: 7006047 (Open Web Text)

Max Activating Token Index: 570

Text #3

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5035. Min Act: -0.1700

Data Index: 3384194 (Open Web Text)

Max Activating Token Index: 71

Text #4

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5130. Min Act: -0.1700

Data Index: 3675549 (Open Web Text)

Max Activating Token Index: 462

Text #5

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.5134. Min Act: -0.1700

Data Index: 7323342 (Open Web Text)

Max Activating Token Index: 784

Text #6

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4855. Min Act: -0.1700

Data Index: 1789856 (Open Web Text)

Max Activating Token Index: 257

Text #7

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4602. Min Act: -0.1700

Data Index: 1916410 (Open Web Text)

Max Activating Token Index: 980

Text #8

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4805. Min Act: -0.1700

Data Index: 5245921 (Open Web Text)

Max Activating Token Index: 943

Text #9

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4898. Min Act: -0.1700

Data Index: 6150601 (Open Web Text)

Max Activating Token Index: 551

Text #10

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4452. Min Act: -0.1700

Data Index: 8528079 (Open Web Text)

Max Activating Token Index: 955

Text #11

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4575. Min Act: -0.1700

Data Index: 6877911 (Open Web Text)

Max Activating Token Index: 731

Text #12

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4465. Min Act: -0.1700

Data Index: 3653158 (Open Web Text)

Max Activating Token Index: 400

Text #13

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4610. Min Act: -0.1700

Data Index: 8627911 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4345. Min Act: -0.1700

Data Index: 7832980 (Open Web Text)

Max Activating Token Index: 987

Text #15

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4403. Min Act: -0.1700

Data Index: 1832250 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4226. Min Act: -0.1700

Data Index: 3299820 (Open Web Text)

Max Activating Token Index: 616

Text #17

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4566. Min Act: -0.1700

Data Index: 4659301 (Open Web Text)

Max Activating Token Index: 923

Text #18

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4642. Min Act: -0.1700

Data Index: 2458512 (Open Web Text)

Max Activating Token Index: 426

Text #19

Max Range: 3.5237. Min Range: -3.5237

Max Act: 3.4604. Min Act: -0.1700

Data Index: 1358017 (Open Web Text)

Max Activating Token Index: 4