Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1980 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.9087. Min Act: -0.1700

Data Index: 1454635 (Open Web Text)

Max Activating Token Index: 825

Text #1

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.5692. Min Act: -0.1700

Data Index: 1217403 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.4748. Min Act: -0.1700

Data Index: 1454636 (Open Web Text)

Max Activating Token Index: 332

Text #3

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.2034. Min Act: -0.1700

Data Index: 5579925 (Open Web Text)

Max Activating Token Index: 349

Text #4

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.1890. Min Act: -0.1700

Data Index: 7695092 (Open Web Text)

Max Activating Token Index: 641

Text #5

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.1282. Min Act: -0.1700

Data Index: 2842344 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.1027. Min Act: -0.1700

Data Index: 2903245 (Open Web Text)

Max Activating Token Index: 846

Text #7

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0941. Min Act: -0.1700

Data Index: 959658 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0820. Min Act: -0.1700

Data Index: 5579919 (Open Web Text)

Max Activating Token Index: 503

Text #9

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0783. Min Act: -0.1700

Data Index: 1259223 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0756. Min Act: -0.1700

Data Index: 4254544 (Open Web Text)

Max Activating Token Index: 253

Text #11

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0550. Min Act: -0.1700

Data Index: 3017300 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0505. Min Act: -0.1700

Data Index: 4938646 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0677. Min Act: -0.1700

Data Index: 4295037 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0544. Min Act: -0.1700

Data Index: 5579917 (Open Web Text)

Max Activating Token Index: 765

Text #15

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0492. Min Act: -0.1700

Data Index: 1059065 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0268. Min Act: -0.1700

Data Index: 4703676 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0435. Min Act: -0.1700

Data Index: 1707582 (Open Web Text)

Max Activating Token Index: 303

Text #18

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0253. Min Act: -0.1700

Data Index: 4292064 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 3.9087. Min Range: -3.9087

Max Act: 3.0317. Min Act: -0.1700

Data Index: 5579927 (Open Web Text)

Max Activating Token Index: 518