Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1078 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.9597. Min Act: -0.1700

Data Index: 1655665 (Open Web Text)

Max Activating Token Index: 243

Text #1

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.8501. Min Act: -0.1700

Data Index: 2177721 (Open Web Text)

Max Activating Token Index: 355

Text #2

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.8553. Min Act: -0.1700

Data Index: 5739122 (Open Web Text)

Max Activating Token Index: 233

Text #3

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.8140. Min Act: -0.1700

Data Index: 8047657 (Open Web Text)

Max Activating Token Index: 773

Text #4

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7799. Min Act: -0.1700

Data Index: 5605054 (Open Web Text)

Max Activating Token Index: 359

Text #5

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7845. Min Act: -0.1700

Data Index: 6521758 (Open Web Text)

Max Activating Token Index: 341

Text #6

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7934. Min Act: -0.1700

Data Index: 6782787 (Open Web Text)

Max Activating Token Index: 384

Text #7

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7219. Min Act: -0.1700

Data Index: 7327333 (Open Web Text)

Max Activating Token Index: 345

Text #8

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7148. Min Act: -0.1700

Data Index: 1453251 (Open Web Text)

Max Activating Token Index: 571

Text #9

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.7002. Min Act: -0.1700

Data Index: 3905952 (Open Web Text)

Max Activating Token Index: 399

Text #10

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6769. Min Act: -0.1700

Data Index: 7132351 (Open Web Text)

Max Activating Token Index: 113

Text #11

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6769. Min Act: -0.1700

Data Index: 3376951 (Open Web Text)

Max Activating Token Index: 382

Text #12

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6782. Min Act: -0.1700

Data Index: 3018080 (Open Web Text)

Max Activating Token Index: 403

Text #13

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6921. Min Act: -0.1700

Data Index: 268247 (Open Web Text)

Max Activating Token Index: 103

Text #14

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6744. Min Act: -0.1700

Data Index: 99521 (Open Web Text)

Max Activating Token Index: 412

Text #15

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6579. Min Act: -0.1700

Data Index: 7678448 (Open Web Text)

Max Activating Token Index: 257

Text #16

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6376. Min Act: -0.1700

Data Index: 7279498 (Open Web Text)

Max Activating Token Index: 914

Text #17

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6720. Min Act: -0.1700

Data Index: 5125081 (Open Web Text)

Max Activating Token Index: 693

Text #18

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6358. Min Act: -0.1700

Data Index: 7084893 (Open Web Text)

Max Activating Token Index: 816

Text #19

Max Range: 5.9597. Min Range: -5.9597

Max Act: 5.6396. Min Act: -0.1700

Data Index: 3014693 (Open Web Text)

Max Activating Token Index: 725