Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1681 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.7940. Min Act: -0.1700

Data Index: 3975886 (Open Web Text)

Max Activating Token Index: 755

Text #1

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.6945. Min Act: -0.1700

Data Index: 1376477 (Open Web Text)

Max Activating Token Index: 67

Text #2

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.6469. Min Act: -0.1700

Data Index: 592115 (Open Web Text)

Max Activating Token Index: 549

Text #3

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.6544. Min Act: -0.1700

Data Index: 1855310 (Open Web Text)

Max Activating Token Index: 330

Text #4

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.6044. Min Act: -0.1700

Data Index: 996478 (Open Web Text)

Max Activating Token Index: 603

Text #5

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.6125. Min Act: -0.1700

Data Index: 4178400 (Open Web Text)

Max Activating Token Index: 122

Text #6

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5470. Min Act: -0.1700

Data Index: 4932339 (Open Web Text)

Max Activating Token Index: 492

Text #7

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5542. Min Act: -0.1700

Data Index: 6757769 (Open Web Text)

Max Activating Token Index: 363

Text #8

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5568. Min Act: -0.1700

Data Index: 4678962 (Open Web Text)

Max Activating Token Index: 122

Text #9

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5443. Min Act: -0.1700

Data Index: 4010850 (Open Web Text)

Max Activating Token Index: 412

Text #10

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5325. Min Act: -0.1700

Data Index: 5814071 (Open Web Text)

Max Activating Token Index: 100

Text #11

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5096. Min Act: -0.1700

Data Index: 6794062 (Open Web Text)

Max Activating Token Index: 682

Text #12

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.4748. Min Act: -0.1700

Data Index: 5431122 (Open Web Text)

Max Activating Token Index: 452

Text #13

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5354. Min Act: -0.1700

Data Index: 4793034 (Open Web Text)

Max Activating Token Index: 150

Text #14

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5193. Min Act: -0.1700

Data Index: 7368889 (Open Web Text)

Max Activating Token Index: 259

Text #15

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.4771. Min Act: -0.1700

Data Index: 950404 (Open Web Text)

Max Activating Token Index: 77

Text #16

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.4872. Min Act: -0.1700

Data Index: 1429068 (Open Web Text)

Max Activating Token Index: 54

Text #17

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.4847. Min Act: -0.1700

Data Index: 7167897 (Open Web Text)

Max Activating Token Index: 535

Text #18

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.5022. Min Act: -0.1700

Data Index: 4823387 (Open Web Text)

Max Activating Token Index: 311

Text #19

Max Range: 4.7940. Min Range: -4.7940

Max Act: 4.4434. Min Act: -0.1700

Data Index: 4781792 (Open Web Text)

Max Activating Token Index: 987