Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 974 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.2100. Min Act: -0.1700

Data Index: 214203 (Open Web Text)

Max Activating Token Index: 132

Text #1

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.1304. Min Act: -0.1700

Data Index: 4291117 (Open Web Text)

Max Activating Token Index: 10

Text #2

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.1256. Min Act: -0.1700

Data Index: 4291119 (Open Web Text)

Max Activating Token Index: 58

Text #3

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0919. Min Act: -0.1700

Data Index: 535510 (Open Web Text)

Max Activating Token Index: 168

Text #4

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0612. Min Act: -0.1700

Data Index: 6514472 (Open Web Text)

Max Activating Token Index: 730

Text #5

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0617. Min Act: -0.1700

Data Index: 4628529 (Open Web Text)

Max Activating Token Index: 426

Text #6

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0468. Min Act: -0.1700

Data Index: 7030664 (Open Web Text)

Max Activating Token Index: 535

Text #7

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0489. Min Act: -0.1700

Data Index: 4302078 (Open Web Text)

Max Activating Token Index: 489

Text #8

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0579. Min Act: -0.1700

Data Index: 3617693 (Open Web Text)

Max Activating Token Index: 12

Text #9

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0624. Min Act: -0.1700

Data Index: 3086175 (Open Web Text)

Max Activating Token Index: 245

Text #10

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0531. Min Act: -0.1700

Data Index: 483203 (Open Web Text)

Max Activating Token Index: 287

Text #11

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9764. Min Act: -0.1700

Data Index: 5714526 (Open Web Text)

Max Activating Token Index: 694

Text #12

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0314. Min Act: -0.1700

Data Index: 3706956 (Open Web Text)

Max Activating Token Index: 156

Text #13

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0297. Min Act: -0.1700

Data Index: 2725596 (Open Web Text)

Max Activating Token Index: 663

Text #14

Max Range: 5.2100. Min Range: -5.2100

Max Act: 5.0010. Min Act: -0.1700

Data Index: 4305881 (Open Web Text)

Max Activating Token Index: 207

Text #15

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9798. Min Act: -0.1700

Data Index: 3506720 (Open Web Text)

Max Activating Token Index: 865

Text #16

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9974. Min Act: -0.1700

Data Index: 3249157 (Open Web Text)

Max Activating Token Index: 393

Text #17

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9895. Min Act: -0.1700

Data Index: 5147855 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9410. Min Act: -0.1700

Data Index: 3512996 (Open Web Text)

Max Activating Token Index: 566

Text #19

Max Range: 5.2100. Min Range: -5.2100

Max Act: 4.9823. Min Act: -0.1700

Data Index: 6408047 (Open Web Text)

Max Activating Token Index: 573