Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1236 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.3966. Min Act: -0.1700

Data Index: 677438 (C4 (Web Text))

Max Activating Token Index: 513

Text #1

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.3204. Min Act: -0.1700

Data Index: 438724 (C4 (Web Text))

Max Activating Token Index: 820

Text #2

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.2981. Min Act: -0.1700

Data Index: 187155 (C4 (Web Text))

Max Activating Token Index: 750

Text #3

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.1983. Min Act: -0.1700

Data Index: 1100974 (C4 (Web Text))

Max Activating Token Index: 424

Text #4

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.1768. Min Act: -0.1699

Data Index: 664943 (C4 (Web Text))

Max Activating Token Index: 798

Text #5

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.1461. Min Act: -0.1696

Data Index: 2538 (C4 (Web Text))

Max Activating Token Index: 418

Text #6

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.1421. Min Act: -0.1700

Data Index: 402930 (C4 (Web Text))

Max Activating Token Index: 708

Text #7

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0988. Min Act: -0.1700

Data Index: 432083 (C4 (Web Text))

Max Activating Token Index: 837

Text #8

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0871. Min Act: -0.1700

Data Index: 708841 (C4 (Web Text))

Max Activating Token Index: 852

Text #9

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0872. Min Act: -0.1700

Data Index: 62640 (C4 (Web Text))

Max Activating Token Index: 591

Text #10

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0847. Min Act: -0.1700

Data Index: 1100167 (C4 (Web Text))

Max Activating Token Index: 494

Text #11

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0637. Min Act: -0.1700

Data Index: 310973 (C4 (Web Text))

Max Activating Token Index: 883

Text #12

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0705. Min Act: -0.1700

Data Index: 282536 (C4 (Web Text))

Max Activating Token Index: 127

Text #13

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0623. Min Act: -0.1700

Data Index: 1628933 (Python Code)

Max Activating Token Index: 670

Text #14

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0671. Min Act: -0.1700

Data Index: 461930 (C4 (Web Text))

Max Activating Token Index: 872

Text #15

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0428. Min Act: -0.1700

Data Index: 813683 (C4 (Web Text))

Max Activating Token Index: 622

Text #16

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0462. Min Act: -0.1700

Data Index: 606835 (C4 (Web Text))

Max Activating Token Index: 214

Text #17

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0293. Min Act: -0.1700

Data Index: 236172 (C4 (Web Text))

Max Activating Token Index: 653

Text #18

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0144. Min Act: -0.1700

Data Index: 224593 (C4 (Web Text))

Max Activating Token Index: 95

Text #19

Max Range: 3.3966. Min Range: -3.3966

Max Act: 3.0044. Min Act: -0.1700

Data Index: 464126 (C4 (Web Text))

Max Activating Token Index: 516