Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2333 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.3133. Min Act: -0.1700

Data Index: 216001 (Open Web Text)

Max Activating Token Index: 993

Text #1

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.2248. Min Act: -0.1700

Data Index: 562783 (Open Web Text)

Max Activating Token Index: 89

Text #2

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.2108. Min Act: -0.1700

Data Index: 6834474 (Open Web Text)

Max Activating Token Index: 548

Text #3

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.2224. Min Act: -0.1700

Data Index: 7186698 (Open Web Text)

Max Activating Token Index: 818

Text #4

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.2121. Min Act: -0.1700

Data Index: 724549 (Open Web Text)

Max Activating Token Index: 757

Text #5

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1787. Min Act: -0.1700

Data Index: 2193526 (Open Web Text)

Max Activating Token Index: 931

Text #6

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1720. Min Act: -0.1700

Data Index: 4846536 (Open Web Text)

Max Activating Token Index: 952

Text #7

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1514. Min Act: -0.1700

Data Index: 3969819 (Open Web Text)

Max Activating Token Index: 704

Text #8

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1660. Min Act: -0.1700

Data Index: 7340445 (Open Web Text)

Max Activating Token Index: 735

Text #9

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1220. Min Act: -0.1700

Data Index: 4186662 (Open Web Text)

Max Activating Token Index: 515

Text #10

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1520. Min Act: -0.1700

Data Index: 3050112 (Open Web Text)

Max Activating Token Index: 249

Text #11

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1385. Min Act: -0.1700

Data Index: 1052958 (Open Web Text)

Max Activating Token Index: 247

Text #12

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1232. Min Act: -0.1700

Data Index: 5700593 (Open Web Text)

Max Activating Token Index: 753

Text #13

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1491. Min Act: -0.1700

Data Index: 762714 (Open Web Text)

Max Activating Token Index: 844

Text #14

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1090. Min Act: -0.1700

Data Index: 4624321 (Open Web Text)

Max Activating Token Index: 368

Text #15

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1045. Min Act: -0.1700

Data Index: 822207 (Open Web Text)

Max Activating Token Index: 726

Text #16

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1247. Min Act: -0.1700

Data Index: 5603331 (Open Web Text)

Max Activating Token Index: 936

Text #17

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1123. Min Act: -0.1700

Data Index: 3088494 (Open Web Text)

Max Activating Token Index: 369

Text #18

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1075. Min Act: -0.1700

Data Index: 6572254 (Open Web Text)

Max Activating Token Index: 568

Text #19

Max Range: 3.3133. Min Range: -3.3133

Max Act: 3.1270. Min Act: -0.1700

Data Index: 5827604 (Open Web Text)

Max Activating Token Index: 640