Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1239 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.4212. Min Act: -0.1700

Data Index: 4316688 (Open Web Text)

Max Activating Token Index: 557

Text #1

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.3636. Min Act: -0.1700

Data Index: 4887945 (Open Web Text)

Max Activating Token Index: 658

Text #2

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2741. Min Act: -0.1700

Data Index: 6937530 (Open Web Text)

Max Activating Token Index: 50

Text #3

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2558. Min Act: -0.1700

Data Index: 369247 (Open Web Text)

Max Activating Token Index: 997

Text #4

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2895. Min Act: -0.1700

Data Index: 3474326 (Open Web Text)

Max Activating Token Index: 502

Text #5

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2519. Min Act: -0.1700

Data Index: 3261013 (Open Web Text)

Max Activating Token Index: 303

Text #6

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2617. Min Act: -0.1700

Data Index: 2296559 (Open Web Text)

Max Activating Token Index: 953

Text #7

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2244. Min Act: -0.1700

Data Index: 8285977 (Open Web Text)

Max Activating Token Index: 455

Text #8

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2467. Min Act: -0.1700

Data Index: 5418804 (Open Web Text)

Max Activating Token Index: 211

Text #9

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2379. Min Act: -0.1700

Data Index: 6441939 (Open Web Text)

Max Activating Token Index: 577

Text #10

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2342. Min Act: -0.1700

Data Index: 1711670 (Open Web Text)

Max Activating Token Index: 235

Text #11

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2424. Min Act: -0.1700

Data Index: 6725805 (Open Web Text)

Max Activating Token Index: 444

Text #12

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2230. Min Act: -0.1700

Data Index: 3381823 (Open Web Text)

Max Activating Token Index: 405

Text #13

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.1955. Min Act: -0.1700

Data Index: 2071062 (Open Web Text)

Max Activating Token Index: 222

Text #14

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2155. Min Act: -0.1700

Data Index: 1910060 (Open Web Text)

Max Activating Token Index: 82

Text #15

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2130. Min Act: -0.1700

Data Index: 4878202 (Open Web Text)

Max Activating Token Index: 771

Text #16

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2351. Min Act: -0.1700

Data Index: 1636916 (Open Web Text)

Max Activating Token Index: 723

Text #17

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2118. Min Act: -0.1700

Data Index: 5530043 (Open Web Text)

Max Activating Token Index: 197

Text #18

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2038. Min Act: -0.1700

Data Index: 6308328 (Open Web Text)

Max Activating Token Index: 764

Text #19

Max Range: 5.4212. Min Range: -5.4212

Max Act: 5.2037. Min Act: -0.1700

Data Index: 6581826 (Open Web Text)

Max Activating Token Index: 114