Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1131 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.4248. Min Act: -0.1700

Data Index: 1853662 (Open Web Text)

Max Activating Token Index: 805

Text #1

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.3975. Min Act: -0.1700

Data Index: 6290797 (Open Web Text)

Max Activating Token Index: 1019

Text #2

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.2999. Min Act: -0.1700

Data Index: 6472430 (Open Web Text)

Max Activating Token Index: 107

Text #3

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1411. Min Act: -0.1700

Data Index: 3972320 (Open Web Text)

Max Activating Token Index: 647

Text #4

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1930. Min Act: -0.1700

Data Index: 4265640 (Open Web Text)

Max Activating Token Index: 813

Text #5

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.2780. Min Act: -0.1700

Data Index: 1010312 (Open Web Text)

Max Activating Token Index: 36

Text #6

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.0842. Min Act: -0.1700

Data Index: 1332075 (Open Web Text)

Max Activating Token Index: 740

Text #7

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1604. Min Act: -0.1700

Data Index: 1311422 (Open Web Text)

Max Activating Token Index: 644

Text #8

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1345. Min Act: -0.1700

Data Index: 4782470 (Open Web Text)

Max Activating Token Index: 563

Text #9

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1820. Min Act: -0.1700

Data Index: 2827274 (Open Web Text)

Max Activating Token Index: 63

Text #10

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1456. Min Act: -0.1700

Data Index: 4628404 (Open Web Text)

Max Activating Token Index: 948

Text #11

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1568. Min Act: -0.1700

Data Index: 3543217 (Open Web Text)

Max Activating Token Index: 968

Text #12

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1589. Min Act: -0.1700

Data Index: 394034 (Open Web Text)

Max Activating Token Index: 757

Text #13

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1107. Min Act: -0.1700

Data Index: 6921981 (Open Web Text)

Max Activating Token Index: 17

Text #14

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1126. Min Act: -0.1700

Data Index: 2334138 (Open Web Text)

Max Activating Token Index: 736

Text #15

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.0632. Min Act: -0.1700

Data Index: 1170652 (Open Web Text)

Max Activating Token Index: 27

Text #16

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.0499. Min Act: -0.1700

Data Index: 6764817 (Open Web Text)

Max Activating Token Index: 495

Text #17

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.0569. Min Act: -0.1700

Data Index: 7099289 (Open Web Text)

Max Activating Token Index: 947

Text #18

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.1019. Min Act: -0.1700

Data Index: 408580 (Open Web Text)

Max Activating Token Index: 613

Text #19

Max Range: 5.4248. Min Range: -5.4248

Max Act: 5.0803. Min Act: -0.1700

Data Index: 3361892 (Open Web Text)

Max Activating Token Index: 702