Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2682 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4754. Min Act: -0.1700

Data Index: 7978558 (Open Web Text)

Max Activating Token Index: 739

Text #1

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4522. Min Act: -0.1700

Data Index: 7737084 (Open Web Text)

Max Activating Token Index: 297

Text #2

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4520. Min Act: -0.1700

Data Index: 7977285 (Open Web Text)

Max Activating Token Index: 320

Text #3

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4305. Min Act: -0.1700

Data Index: 3352028 (Open Web Text)

Max Activating Token Index: 724

Text #4

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4140. Min Act: -0.1700

Data Index: 5343938 (Open Web Text)

Max Activating Token Index: 516

Text #5

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.4280. Min Act: -0.1700

Data Index: 6340282 (Open Web Text)

Max Activating Token Index: 429

Text #6

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3756. Min Act: -0.1700

Data Index: 2944908 (Open Web Text)

Max Activating Token Index: 330

Text #7

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3586. Min Act: -0.1700

Data Index: 4574773 (Open Web Text)

Max Activating Token Index: 1009

Text #8

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3851. Min Act: -0.1700

Data Index: 1883273 (Open Web Text)

Max Activating Token Index: 157

Text #9

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3682. Min Act: -0.1700

Data Index: 8507639 (Open Web Text)

Max Activating Token Index: 786

Text #10

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3375. Min Act: -0.1700

Data Index: 7295537 (Open Web Text)

Max Activating Token Index: 474

Text #11

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3775. Min Act: -0.1700

Data Index: 506442 (Open Web Text)

Max Activating Token Index: 394

Text #12

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3114. Min Act: -0.1700

Data Index: 4195811 (Open Web Text)

Max Activating Token Index: 632

Text #13

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3206. Min Act: -0.1700

Data Index: 706237 (Open Web Text)

Max Activating Token Index: 989

Text #14

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3074. Min Act: -0.1700

Data Index: 1404061 (Open Web Text)

Max Activating Token Index: 773

Text #15

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3116. Min Act: -0.1700

Data Index: 5883933 (Open Web Text)

Max Activating Token Index: 550

Text #16

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3270. Min Act: -0.1700

Data Index: 5280101 (Open Web Text)

Max Activating Token Index: 592

Text #17

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.3284. Min Act: -0.1700

Data Index: 2993811 (Open Web Text)

Max Activating Token Index: 1013

Text #18

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.2985. Min Act: -0.1700

Data Index: 4146605 (Open Web Text)

Max Activating Token Index: 990

Text #19

Max Range: 3.4754. Min Range: -3.4754

Max Act: 3.2980. Min Act: -0.1700

Data Index: 3763066 (Open Web Text)

Max Activating Token Index: 996