Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 891 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.2764. Min Act: -0.1700

Data Index: 896942 (Open Web Text)

Max Activating Token Index: 1007

Text #1

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1980. Min Act: -0.1700

Data Index: 8787492 (Open Web Text)

Max Activating Token Index: 210

Text #2

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.2061. Min Act: -0.1700

Data Index: 8726000 (Open Web Text)

Max Activating Token Index: 391

Text #3

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1904. Min Act: -0.1700

Data Index: 3680137 (Open Web Text)

Max Activating Token Index: 1001

Text #4

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1907. Min Act: -0.1700

Data Index: 7302731 (Open Web Text)

Max Activating Token Index: 454

Text #5

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1535. Min Act: -0.1700

Data Index: 3850178 (Open Web Text)

Max Activating Token Index: 238

Text #6

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1544. Min Act: -0.1700

Data Index: 7567100 (Open Web Text)

Max Activating Token Index: 898

Text #7

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1460. Min Act: -0.1700

Data Index: 6155743 (Open Web Text)

Max Activating Token Index: 937

Text #8

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1382. Min Act: -0.1700

Data Index: 3387575 (Open Web Text)

Max Activating Token Index: 273

Text #9

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1526. Min Act: -0.1700

Data Index: 943701 (Open Web Text)

Max Activating Token Index: 976

Text #10

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1388. Min Act: -0.1700

Data Index: 3316294 (Open Web Text)

Max Activating Token Index: 172

Text #11

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.1180. Min Act: -0.1700

Data Index: 3491054 (Open Web Text)

Max Activating Token Index: 142

Text #12

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0946. Min Act: -0.1700

Data Index: 6317570 (Open Web Text)

Max Activating Token Index: 548

Text #13

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0983. Min Act: -0.1700

Data Index: 6996188 (Open Web Text)

Max Activating Token Index: 576

Text #14

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0708. Min Act: -0.1700

Data Index: 6217190 (Open Web Text)

Max Activating Token Index: 805

Text #15

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0797. Min Act: -0.1700

Data Index: 6266895 (Open Web Text)

Max Activating Token Index: 507

Text #16

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0957. Min Act: -0.1700

Data Index: 7423165 (Open Web Text)

Max Activating Token Index: 532

Text #17

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0961. Min Act: -0.1700

Data Index: 4714492 (Open Web Text)

Max Activating Token Index: 856

Text #18

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0644. Min Act: -0.1700

Data Index: 3910472 (Open Web Text)

Max Activating Token Index: 232

Text #19

Max Range: 3.2764. Min Range: -3.2764

Max Act: 3.0677. Min Act: -0.1700

Data Index: 2245497 (Open Web Text)

Max Activating Token Index: 469