Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2687 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.6746. Min Act: -0.1700

Data Index: 3875194 (Open Web Text)

Max Activating Token Index: 348

Text #1

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.6262. Min Act: -0.1700

Data Index: 6729345 (Open Web Text)

Max Activating Token Index: 94

Text #2

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.6027. Min Act: -0.1700

Data Index: 2123544 (Open Web Text)

Max Activating Token Index: 116

Text #3

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.6017. Min Act: -0.1700

Data Index: 3717611 (Open Web Text)

Max Activating Token Index: 755

Text #4

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.6225. Min Act: -0.1700

Data Index: 5485978 (Open Web Text)

Max Activating Token Index: 103

Text #5

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5719. Min Act: -0.1700

Data Index: 395062 (Open Web Text)

Max Activating Token Index: 680

Text #6

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5752. Min Act: -0.1700

Data Index: 7696146 (Open Web Text)

Max Activating Token Index: 704

Text #7

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5483. Min Act: -0.1700

Data Index: 5687290 (Open Web Text)

Max Activating Token Index: 883

Text #8

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5427. Min Act: -0.1700

Data Index: 8268289 (Open Web Text)

Max Activating Token Index: 159

Text #9

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5220. Min Act: -0.1700

Data Index: 3510075 (Open Web Text)

Max Activating Token Index: 853

Text #10

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5237. Min Act: -0.1700

Data Index: 4687267 (Open Web Text)

Max Activating Token Index: 743

Text #11

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5330. Min Act: -0.1700

Data Index: 721968 (Open Web Text)

Max Activating Token Index: 180

Text #12

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5405. Min Act: -0.1700

Data Index: 3837806 (Open Web Text)

Max Activating Token Index: 846

Text #13

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5435. Min Act: -0.1700

Data Index: 4948184 (Open Web Text)

Max Activating Token Index: 89

Text #14

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5388. Min Act: -0.1700

Data Index: 4390510 (Open Web Text)

Max Activating Token Index: 21

Text #15

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5077. Min Act: -0.1700

Data Index: 173212 (Open Web Text)

Max Activating Token Index: 632

Text #16

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5136. Min Act: -0.1700

Data Index: 4960417 (Open Web Text)

Max Activating Token Index: 548

Text #17

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5199. Min Act: -0.1700

Data Index: 314366 (Open Web Text)

Max Activating Token Index: 288

Text #18

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.5048. Min Act: -0.1700

Data Index: 8335480 (Open Web Text)

Max Activating Token Index: 660

Text #19

Max Range: 3.6746. Min Range: -3.6746

Max Act: 3.4931. Min Act: -0.1700

Data Index: 8697093 (Open Web Text)

Max Activating Token Index: 197