Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2798 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4736. Min Act: -0.1700

Data Index: 5040141 (Open Web Text)

Max Activating Token Index: 708

Text #1

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4537. Min Act: -0.1700

Data Index: 6629129 (Open Web Text)

Max Activating Token Index: 69

Text #2

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4199. Min Act: -0.1700

Data Index: 2233216 (Open Web Text)

Max Activating Token Index: 690

Text #3

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4292. Min Act: -0.1700

Data Index: 4653122 (Open Web Text)

Max Activating Token Index: 795

Text #4

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4329. Min Act: -0.1700

Data Index: 4886947 (Open Web Text)

Max Activating Token Index: 812

Text #5

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3998. Min Act: -0.1700

Data Index: 4714121 (Open Web Text)

Max Activating Token Index: 724

Text #6

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4095. Min Act: -0.1700

Data Index: 5798898 (Open Web Text)

Max Activating Token Index: 833

Text #7

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4064. Min Act: -0.1700

Data Index: 7809449 (Open Web Text)

Max Activating Token Index: 331

Text #8

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4012. Min Act: -0.1700

Data Index: 2908562 (Open Web Text)

Max Activating Token Index: 857

Text #9

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3951. Min Act: -0.1700

Data Index: 800576 (Open Web Text)

Max Activating Token Index: 668

Text #10

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3931. Min Act: -0.1700

Data Index: 1525199 (Open Web Text)

Max Activating Token Index: 92

Text #11

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.4050. Min Act: -0.1700

Data Index: 7348332 (Open Web Text)

Max Activating Token Index: 201

Text #12

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3729. Min Act: -0.1700

Data Index: 811552 (Open Web Text)

Max Activating Token Index: 202

Text #13

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3742. Min Act: -0.1700

Data Index: 7778657 (Open Web Text)

Max Activating Token Index: 609

Text #14

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3606. Min Act: -0.1700

Data Index: 7068589 (Open Web Text)

Max Activating Token Index: 243

Text #15

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3619. Min Act: -0.1700

Data Index: 8007028 (Open Web Text)

Max Activating Token Index: 671

Text #16

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3569. Min Act: -0.1700

Data Index: 3909774 (Open Web Text)

Max Activating Token Index: 62

Text #17

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3503. Min Act: -0.1700

Data Index: 5941475 (Open Web Text)

Max Activating Token Index: 160

Text #18

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3539. Min Act: -0.1700

Data Index: 3096549 (Open Web Text)

Max Activating Token Index: 637

Text #19

Max Range: 3.4736. Min Range: -3.4736

Max Act: 3.3552. Min Act: -0.1700

Data Index: 6086758 (Open Web Text)

Max Activating Token Index: 224