Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 293 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4785. Min Act: -0.1700

Data Index: 3296300 (Open Web Text)

Max Activating Token Index: 417

Text #1

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4680. Min Act: -0.1700

Data Index: 1825013 (Open Web Text)

Max Activating Token Index: 725

Text #2

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4765. Min Act: -0.1699

Data Index: 8781322 (Open Web Text)

Max Activating Token Index: 336

Text #3

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4784. Min Act: -0.1700

Data Index: 8498739 (Open Web Text)

Max Activating Token Index: 498

Text #4

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4744. Min Act: -0.1700

Data Index: 6052343 (Open Web Text)

Max Activating Token Index: 303

Text #5

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4813. Min Act: -0.1700

Data Index: 6216828 (Open Web Text)

Max Activating Token Index: 316

Text #6

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4830. Min Act: -0.1700

Data Index: 6288502 (Open Web Text)

Max Activating Token Index: 497

Text #7

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4622. Min Act: -0.1700

Data Index: 4750046 (Open Web Text)

Max Activating Token Index: 528

Text #8

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4645. Min Act: -0.1700

Data Index: 6446410 (Open Web Text)

Max Activating Token Index: 481

Text #9

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4698. Min Act: -0.1700

Data Index: 5340120 (Open Web Text)

Max Activating Token Index: 535

Text #10

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4577. Min Act: -0.1700

Data Index: 3175108 (Open Web Text)

Max Activating Token Index: 169

Text #11

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4540. Min Act: -0.1700

Data Index: 403414 (Open Web Text)

Max Activating Token Index: 692

Text #12

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4584. Min Act: -0.1700

Data Index: 4444541 (Open Web Text)

Max Activating Token Index: 436

Text #13

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4512. Min Act: -0.1700

Data Index: 5882736 (Open Web Text)

Max Activating Token Index: 279

Text #14

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4537. Min Act: -0.1700

Data Index: 6778210 (Open Web Text)

Max Activating Token Index: 803

Text #15

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4570. Min Act: -0.1699

Data Index: 1853832 (Open Web Text)

Max Activating Token Index: 949

Text #16

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4410. Min Act: -0.1700

Data Index: 531022 (Open Web Text)

Max Activating Token Index: 170

Text #17

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4570. Min Act: -0.1699

Data Index: 3120564 (Open Web Text)

Max Activating Token Index: 411

Text #18

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4460. Min Act: -0.1700

Data Index: 2512111 (Open Web Text)

Max Activating Token Index: 165

Text #19

Max Range: 1.4830. Min Range: -1.4830

Max Act: 1.4483. Min Act: -0.1700

Data Index: 5807907 (Open Web Text)

Max Activating Token Index: 582