Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1279 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0294. Min Range: -3.0294

Max Act: 3.0294. Min Act: -0.1700

Data Index: 8447809 (Open Web Text)

Max Activating Token Index: 506

Text #1

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.9159. Min Act: -0.1700

Data Index: 7726947 (Open Web Text)

Max Activating Token Index: 133

Text #2

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8860. Min Act: -0.1700

Data Index: 4381762 (Open Web Text)

Max Activating Token Index: 812

Text #3

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8756. Min Act: -0.1700

Data Index: 4277471 (Open Web Text)

Max Activating Token Index: 428

Text #4

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8753. Min Act: -0.1700

Data Index: 1587955 (Open Web Text)

Max Activating Token Index: 171

Text #5

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8222. Min Act: -0.1700

Data Index: 3634047 (Open Web Text)

Max Activating Token Index: 63

Text #6

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8108. Min Act: -0.1700

Data Index: 1471857 (Open Web Text)

Max Activating Token Index: 482

Text #7

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7756. Min Act: -0.1700

Data Index: 3732717 (Open Web Text)

Max Activating Token Index: 595

Text #8

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7407. Min Act: -0.1700

Data Index: 7490580 (Open Web Text)

Max Activating Token Index: 98

Text #9

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7875. Min Act: -0.1700

Data Index: 665426 (Open Web Text)

Max Activating Token Index: 580

Text #10

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.6878. Min Act: -0.1700

Data Index: 1474447 (Open Web Text)

Max Activating Token Index: 893

Text #11

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.8005. Min Act: -0.1700

Data Index: 1931529 (Open Web Text)

Max Activating Token Index: 528

Text #12

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7674. Min Act: -0.1700

Data Index: 8303163 (Open Web Text)

Max Activating Token Index: 608

Text #13

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7165. Min Act: -0.1700

Data Index: 2521625 (Open Web Text)

Max Activating Token Index: 101

Text #14

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7176. Min Act: -0.1700

Data Index: 5072642 (Open Web Text)

Max Activating Token Index: 665

Text #15

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.6552. Min Act: -0.1700

Data Index: 7649931 (Open Web Text)

Max Activating Token Index: 674

Text #16

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7141. Min Act: -0.1700

Data Index: 2276708 (Open Web Text)

Max Activating Token Index: 687

Text #17

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7082. Min Act: -0.1700

Data Index: 3382524 (Open Web Text)

Max Activating Token Index: 688

Text #18

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7037. Min Act: -0.1700

Data Index: 7892405 (Open Web Text)

Max Activating Token Index: 139

Text #19

Max Range: 3.0294. Min Range: -3.0294

Max Act: 2.7499. Min Act: -0.1700

Data Index: 7940386 (Open Web Text)

Max Activating Token Index: 116