Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2313 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2372. Min Range: -5.2372

Max Act: 5.2372. Min Act: -0.1700

Data Index: 2324678 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.6342. Min Act: -0.1700

Data Index: 8312664 (Open Web Text)

Max Activating Token Index: 487

Text #2

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.6487. Min Act: -0.1700

Data Index: 1602832 (Open Web Text)

Max Activating Token Index: 225

Text #3

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.6058. Min Act: -0.1700

Data Index: 17413 (Open Web Text)

Max Activating Token Index: 765

Text #4

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.6315. Min Act: -0.1700

Data Index: 3454840 (Open Web Text)

Max Activating Token Index: 1000

Text #5

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.5850. Min Act: -0.1700

Data Index: 4130930 (Open Web Text)

Max Activating Token Index: 457

Text #6

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.5744. Min Act: -0.1700

Data Index: 242384 (Open Web Text)

Max Activating Token Index: 29

Text #7

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.5427. Min Act: -0.1700

Data Index: 6683421 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.5163. Min Act: -0.1700

Data Index: 505569 (Open Web Text)

Max Activating Token Index: 586

Text #9

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4887. Min Act: -0.1700

Data Index: 7766923 (Open Web Text)

Max Activating Token Index: 323

Text #10

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4951. Min Act: -0.1700

Data Index: 2158535 (Open Web Text)

Max Activating Token Index: 304

Text #11

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4802. Min Act: -0.1700

Data Index: 6785326 (Open Web Text)

Max Activating Token Index: 241

Text #12

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4974. Min Act: -0.1700

Data Index: 5700289 (Open Web Text)

Max Activating Token Index: 864

Text #13

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4575. Min Act: -0.1700

Data Index: 2046723 (Open Web Text)

Max Activating Token Index: 892

Text #14

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.3977. Min Act: -0.1700

Data Index: 1171363 (Open Web Text)

Max Activating Token Index: 977

Text #15

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4460. Min Act: -0.1700

Data Index: 6142946 (Open Web Text)

Max Activating Token Index: 553

Text #16

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4429. Min Act: -0.1700

Data Index: 2535693 (Open Web Text)

Max Activating Token Index: 318

Text #17

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4509. Min Act: -0.1700

Data Index: 1319659 (Open Web Text)

Max Activating Token Index: 390

Text #18

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4223. Min Act: -0.1700

Data Index: 5427834 (Open Web Text)

Max Activating Token Index: 651

Text #19

Max Range: 5.2372. Min Range: -5.2372

Max Act: 4.4117. Min Act: -0.1700

Data Index: 6636500 (Open Web Text)

Max Activating Token Index: 974