Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2178 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.5403. Min Act: -0.1700

Data Index: 4607367 (Open Web Text)

Max Activating Token Index: 411

Text #1

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.5300. Min Act: -0.1700

Data Index: 6513754 (Open Web Text)

Max Activating Token Index: 711

Text #2

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.5359. Min Act: -0.1700

Data Index: 392830 (Open Web Text)

Max Activating Token Index: 230

Text #3

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4903. Min Act: -0.1700

Data Index: 6641330 (Open Web Text)

Max Activating Token Index: 729

Text #4

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4803. Min Act: -0.1700

Data Index: 8184275 (Open Web Text)

Max Activating Token Index: 348

Text #5

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4896. Min Act: -0.1700

Data Index: 1037560 (Open Web Text)

Max Activating Token Index: 143

Text #6

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4636. Min Act: -0.1700

Data Index: 8514654 (Open Web Text)

Max Activating Token Index: 507

Text #7

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4662. Min Act: -0.1700

Data Index: 5266362 (Open Web Text)

Max Activating Token Index: 240

Text #8

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4618. Min Act: -0.1700

Data Index: 5139105 (Open Web Text)

Max Activating Token Index: 804

Text #9

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4738. Min Act: -0.1700

Data Index: 4500844 (Open Web Text)

Max Activating Token Index: 855

Text #10

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4504. Min Act: -0.1700

Data Index: 1822443 (Open Web Text)

Max Activating Token Index: 998

Text #11

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4685. Min Act: -0.1700

Data Index: 2521739 (Open Web Text)

Max Activating Token Index: 326

Text #12

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4617. Min Act: -0.1700

Data Index: 1612921 (Open Web Text)

Max Activating Token Index: 538

Text #13

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4841. Min Act: -0.1700

Data Index: 6641324 (Open Web Text)

Max Activating Token Index: 377

Text #14

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4631. Min Act: -0.1700

Data Index: 749847 (Open Web Text)

Max Activating Token Index: 190

Text #15

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4599. Min Act: -0.1700

Data Index: 6512043 (Open Web Text)

Max Activating Token Index: 263

Text #16

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4591. Min Act: -0.1700

Data Index: 2858685 (Open Web Text)

Max Activating Token Index: 675

Text #17

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4825. Min Act: -0.1700

Data Index: 5846390 (Open Web Text)

Max Activating Token Index: 677

Text #18

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4497. Min Act: -0.1700

Data Index: 861409 (Open Web Text)

Max Activating Token Index: 603

Text #19

Max Range: 3.5403. Min Range: -3.5403

Max Act: 3.4427. Min Act: -0.1700

Data Index: 109524 (Open Web Text)

Max Activating Token Index: 82