Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1771 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0957. Min Range: -3.0957

Max Act: 3.0957. Min Act: -0.1700

Data Index: 2943345 (Open Web Text)

Max Activating Token Index: 887

Text #1

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.8342. Min Act: -0.1700

Data Index: 116410 (Open Web Text)

Max Activating Token Index: 85

Text #2

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.7358. Min Act: -0.1700

Data Index: 5629519 (Open Web Text)

Max Activating Token Index: 750

Text #3

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.7099. Min Act: -0.1700

Data Index: 954789 (Open Web Text)

Max Activating Token Index: 878

Text #4

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6906. Min Act: -0.1700

Data Index: 7498333 (Open Web Text)

Max Activating Token Index: 185

Text #5

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.7058. Min Act: -0.1700

Data Index: 3833795 (Open Web Text)

Max Activating Token Index: 541

Text #6

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6925. Min Act: -0.1700

Data Index: 1391774 (Open Web Text)

Max Activating Token Index: 746

Text #7

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.7040. Min Act: -0.1700

Data Index: 5684745 (Open Web Text)

Max Activating Token Index: 283

Text #8

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6794. Min Act: -0.1700

Data Index: 4615983 (Open Web Text)

Max Activating Token Index: 611

Text #9

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6877. Min Act: -0.1700

Data Index: 5396338 (Open Web Text)

Max Activating Token Index: 629

Text #10

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6851. Min Act: -0.1700

Data Index: 5713579 (Open Web Text)

Max Activating Token Index: 279

Text #11

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6575. Min Act: -0.1700

Data Index: 7766131 (Open Web Text)

Max Activating Token Index: 938

Text #12

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6670. Min Act: -0.1700

Data Index: 1235284 (Open Web Text)

Max Activating Token Index: 439

Text #13

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6584. Min Act: -0.1700

Data Index: 1640520 (Open Web Text)

Max Activating Token Index: 589

Text #14

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6399. Min Act: -0.1700

Data Index: 6785310 (Open Web Text)

Max Activating Token Index: 655

Text #15

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6258. Min Act: -0.1700

Data Index: 8627575 (Open Web Text)

Max Activating Token Index: 851

Text #16

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6357. Min Act: -0.1700

Data Index: 7853702 (Open Web Text)

Max Activating Token Index: 653

Text #17

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6247. Min Act: -0.1700

Data Index: 7323682 (Open Web Text)

Max Activating Token Index: 173

Text #18

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6214. Min Act: -0.1700

Data Index: 5494784 (Open Web Text)

Max Activating Token Index: 303

Text #19

Max Range: 3.0957. Min Range: -3.0957

Max Act: 2.6502. Min Act: -0.1700

Data Index: 1428749 (Open Web Text)

Max Activating Token Index: 338