Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 357 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.2619. Min Act: -0.1700

Data Index: 3354656 (Open Web Text)

Max Activating Token Index: 987

Text #1

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.1886. Min Act: -0.1700

Data Index: 8091805 (Open Web Text)

Max Activating Token Index: 916

Text #2

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.1577. Min Act: -0.1700

Data Index: 7371795 (Open Web Text)

Max Activating Token Index: 361

Text #3

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.1182. Min Act: -0.1700

Data Index: 4354275 (Open Web Text)

Max Activating Token Index: 798

Text #4

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.0962. Min Act: -0.1700

Data Index: 5549027 (Open Web Text)

Max Activating Token Index: 111

Text #5

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.0225. Min Act: -0.1700

Data Index: 7101018 (Open Web Text)

Max Activating Token Index: 830

Text #6

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9824. Min Act: -0.1700

Data Index: 4531125 (Open Web Text)

Max Activating Token Index: 642

Text #7

Max Range: 5.2619. Min Range: -5.2619

Max Act: 5.0055. Min Act: -0.1700

Data Index: 6500660 (Open Web Text)

Max Activating Token Index: 61

Text #8

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9857. Min Act: -0.1700

Data Index: 1684677 (Open Web Text)

Max Activating Token Index: 139

Text #9

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9600. Min Act: -0.1700

Data Index: 6571427 (Open Web Text)

Max Activating Token Index: 754

Text #10

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9417. Min Act: -0.1700

Data Index: 4381661 (Open Web Text)

Max Activating Token Index: 32

Text #11

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9202. Min Act: -0.1700

Data Index: 7830242 (Open Web Text)

Max Activating Token Index: 785

Text #12

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9340. Min Act: -0.1700

Data Index: 1710363 (Open Web Text)

Max Activating Token Index: 889

Text #13

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8987. Min Act: -0.1700

Data Index: 8538741 (Open Web Text)

Max Activating Token Index: 249

Text #14

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.9064. Min Act: -0.1700

Data Index: 8291456 (Open Web Text)

Max Activating Token Index: 214

Text #15

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8808. Min Act: -0.1700

Data Index: 296971 (Open Web Text)

Max Activating Token Index: 853

Text #16

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8774. Min Act: -0.1700

Data Index: 5393737 (Open Web Text)

Max Activating Token Index: 283

Text #17

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8634. Min Act: -0.1700

Data Index: 4615220 (Open Web Text)

Max Activating Token Index: 580

Text #18

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8662. Min Act: -0.1700

Data Index: 5594230 (Open Web Text)

Max Activating Token Index: 416

Text #19

Max Range: 5.2619. Min Range: -5.2619

Max Act: 4.8880. Min Act: -0.1700

Data Index: 2528924 (Open Web Text)

Max Activating Token Index: 687