Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2472 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.2556. Min Act: -0.1700

Data Index: 6062255 (Open Web Text)

Max Activating Token Index: 472

Text #1

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.2477. Min Act: -0.1700

Data Index: 4465060 (Open Web Text)

Max Activating Token Index: 726

Text #2

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1846. Min Act: -0.1700

Data Index: 4534520 (Open Web Text)

Max Activating Token Index: 848

Text #3

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1578. Min Act: -0.1700

Data Index: 7606719 (Open Web Text)

Max Activating Token Index: 680

Text #4

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1532. Min Act: -0.1700

Data Index: 4776286 (Open Web Text)

Max Activating Token Index: 261

Text #5

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1641. Min Act: -0.1700

Data Index: 5024858 (Open Web Text)

Max Activating Token Index: 853

Text #6

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1424. Min Act: -0.1700

Data Index: 354888 (Open Web Text)

Max Activating Token Index: 944

Text #7

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1314. Min Act: -0.1700

Data Index: 4543197 (Open Web Text)

Max Activating Token Index: 554

Text #8

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1496. Min Act: -0.1700

Data Index: 8577916 (Open Web Text)

Max Activating Token Index: 1015

Text #9

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1131. Min Act: -0.1700

Data Index: 5004663 (Open Web Text)

Max Activating Token Index: 901

Text #10

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1204. Min Act: -0.1700

Data Index: 1492091 (Open Web Text)

Max Activating Token Index: 311

Text #11

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0978. Min Act: -0.1700

Data Index: 5827170 (Open Web Text)

Max Activating Token Index: 735

Text #12

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0990. Min Act: -0.1700

Data Index: 8803486 (Open Web Text)

Max Activating Token Index: 812

Text #13

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0959. Min Act: -0.1700

Data Index: 1909994 (Open Web Text)

Max Activating Token Index: 622

Text #14

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0988. Min Act: -0.1700

Data Index: 1536887 (Open Web Text)

Max Activating Token Index: 646

Text #15

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1119. Min Act: -0.1700

Data Index: 238042 (Open Web Text)

Max Activating Token Index: 965

Text #16

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0937. Min Act: -0.1700

Data Index: 6127868 (Open Web Text)

Max Activating Token Index: 838

Text #17

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.1255. Min Act: -0.1700

Data Index: 900983 (Open Web Text)

Max Activating Token Index: 352

Text #18

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0751. Min Act: -0.1700

Data Index: 5080277 (Open Web Text)

Max Activating Token Index: 189

Text #19

Max Range: 3.2556. Min Range: -3.2556

Max Act: 3.0792. Min Act: -0.1700

Data Index: 3814959 (Open Web Text)

Max Activating Token Index: 507